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NULLIUS IN VERBA: SCIENCE VS PSEUDO-SCIENCE/FRINGE SCIENCE 


Boris Abersek 
University of Maribor, Slovenia 


The Royal Society's! motto ‘Nullius in verba’ is taken to mean ‘take nobody's word for it’. It is an expression of the 
determination of Fellows to withstand the domination of authority and to verify all statements by an appeal to facts 
determined by experiment. 

Royal Society 


The explosion of disinformation about global warming and other ecological problems could be called an 
infodemic?, which is the term used by WHO to describe today’s pandemic situation. An infodemic means too much 
information, however, its importance can also be passed on to environmental problems. It causes confusion and 
risk-taking behaviors that can harm the health of our planet. It also leads to mistrust in authorities and undermines 
the public response. An infodemic can intensify or lengthen outbreaks when people are unsure about what they 
need to do, for example, to protect our environment. With growing digitization — an expansion of social media 
and internet use — information can spread more rapidly. This can help to more quickly fill information voids but 
can also amplify harmful messages. 

Infodemic management is the systematic use of risk- and evidence-based analysis and approaches to man- 
age the infodemic and reduce its impact on our environment. Infodemic management in relation to ecological 
problems focuses on enabling good practices through four types of activities: 


e Listening to community concerns and questions. 

e Promoting understanding of risk and health expert advice. 

e Strengthening resilience to misinformation. 

e Engaging and empowering communities to take positive action. 


Infodemic management is widely known, but much less discussed is the role of ostensible “experts” or fringe 
scientists in perpetuating dangerous fictions. Since the dawn of the crisis, a disconcerting number of eminently 
qualified (pseudo)scientists and physicians have propagated falsehoods across social media, elevating themselves 
to the status of gurus in order to lend a veneer of seeming scientific legitimacy to empty, dangerous claims. And 


1 https://royalsociety.org/about-us/history/ 
2  Aninfodemic is too much information including false or misleading information in digital and physical environments during a disease 
outbreak. See: https://www.who.int/health-topics/infodemic#tab=tab_1 
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these bogus claims, like their pathological namesake, have gone uncontrollably viral. These fringe pseudoscientists 
are lauded as experts unafraid to speak truth to power. But it is crucial to note that these individuals, for all their 
formal credentials, extol a narrative completely at odds with reality, readily refuted by public bodies the world over. 
These pseudoscientific, conspiratorial claims are archetypal arguments from authority, where a perceived expert's 
support is used to justify positions unsupported by data. Scientific claims do not derive their authority by virtue 
of coming from scientists but from the weight of the evidence behind them. Pseudoscience, by contrast, tends 
to focus on ostensible gurus (see Kanneman system 1) rather than a consensus opinion (see Kanneman system 2) 
(Kanneman, 2011). The only authority a scientist can ever truly invoke is a reflected one, dependent on accurately 
representing the evidence base. If they embrace fringe positions and jettison the principles of scientific skepticism, 
then their qualifications, education, and prestige mean absolutely nothing (Grimes, 2021). 

In order to justify facts, scientists (unfortunately) use complex scientific methods, which are largely incom- 
prehensible to most people. Meanwhile, pseudo-scientists use methods that are closer and more unambiguous 
to the everyman. Real science should be more approachable to the everyman, it should use such methods and 
arguments, which would be easier for people to understand, while at the same time not crossing the line of dis- 
missing the accuracy and, above all, the reliability of what is being said, at the expense of improving intelligibility. 
If, while generalizing, we omit the less relevant and difficult-to-understand arguments and generalize to the level 
of the general population’s understanding, then — as we reverse the generalization, adding more or less relevant 
data, which was omitted in the process of simplification - we should always be able to return to the starting point 
of the problem at hand, to the same complex scientific explanation. 

In essence, the simplification process must be reversible, which means that a reverse process will bring us 
back to the starting point, which is symbolically shown in Figure 1. 


Figure 1 
The Simplification Principle (Abersek, 2015, a,b) 
Science-based Simplification Simplified 
representation 1 Simplification Simplification representation 
2 3 
Oe Abolishment || Abolishment |¢1 Abolishment ~«———— 
of of of 
simplification simplification simplification 
1 2 3 


Misrepresentationg+~ 


Misconception 


The Difference between COVID-19 and Environmental Problems 


Though people live in the here-and-now, they have the capacity to think about future events, remote places, 
distant others, and alternative realities. This ability to think about events that extend beyond the immediate context 
is one that many might take for granted. Construal level theory (CLT) attempts to explain how people accomplish 
this remarkable feat. Central to understanding CLT is the idea of psychological distance (Trope & Liberman, 2010). 
Construal level theory (CLT) is a theory in social psychology that describes the relationship between psychological 
distance and the extent to which people's thinking (e.g., about objects and events) is abstract or concrete. The 
general idea is that the more distant an object is from the individual, the more abstract it will be thought of, while 
the closer the object is, the more concretely it will be thought of. 

Let us provide a topical example of alow level construal, i.e., when people think more about the context-specific 
features: COVID-19. We are all aware that we will have to find a solution as soon as possible, and the only currently 
logical solution is vaccination. With this in mind, scientists have joined forces and developed a vaccine in less than 
a year’s time. The process of vaccine development and adoption, which in recent history took considerably longer, 
several years, now took less than a year. This solution, which may be more or less optimal, reliable and hazardous, 
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has withstood the anti-vaccine sentiment of numerous and various groups, and the vast majority of people have 
decided to get vaccinated, which makes perfect sense, considering the state of chaos in which we currently are. 

High level construal means that the more distant an object is from the individual, the more abstractly it will 
be thought of. With long-term abstract thinking, many of the things we are aware of fall away, we negate time, we 
are not ready to make compromises, we doubt the proposed solutions, or even deny the existence of a problem 
(for example, environmental issues), and there is always the question of whom we can trust. The foundation of 
trust is based upon the realization that various “non-scientists and pseudo-scientists” are exploiting science for 
their own purposes, thereby destroying trust in science. The rise of pseudo-experts is perhaps symptomatic of a 
change in how we access information. As we become curators of our own media, the traditional gatekeepers and 
fact-checkers once implicit in most reporting have been increasingly sidelined. This in turn has made us more 
polarized and reduced our ability to differentiate fact from opinion. Motivated reasoning?, our human bias towards 
cherry-picking only arguments that chime with that we wish were true, most certainly plays a role. The imposi- 
tions of global warming and other ecological problems are manifold; it is not surprising that fringe scientists are 
inevitably invoked as sources for those with strong feelings against, for example, measures to reduce greenhouse 
emissions. Even if we are not ideologically predisposed to such positions, these claims undermine public under- 
standing, blurring perceptions of scientific consensus, nudging us collectively towards fear and distrust. 

Understanding the process of science can protect people against misinformation — or at least we hope so. 
One of the most intriguing stages in the process of science is noticing when something is ... weird. 2020 has beena 
historic year — and mostly not in a good way. Among many things, we saw a historic level of disregard of scientific 
advice with respect to the COVID-19 virus. But while the events of 2020 may feel unprecedented, the social pat- 
tern of rejecting scientific evidence did not suddenly appear this year. There was never any good scientific reason 
for rejecting the expert advice on COVID, just as there has never been any good scientific reason for doubting 
that humans evolved, that vaccines save lives, and that greenhouse gases are driving disruptive climate change. 

For all their qualifications, fringe scientists fail this basic tenet of science, as they are united in their willing- 
ness to embrace conspiracy theory when their claims are refuted. Lack of evidence for their position is airily 
dismissed as a cover-up by everyone from authorities at a global level, to the entire ecological establishment. But 
this performative outrage is so much sound and fury to distract from the inescapable reality that their positions 
are completely contradicted by the overwhelming weight of scientific evidence. This is scientifically reprehensible, 
and staggeringly irresponsible, conduct. 


Potential Solutions 


In recent decades, the gap between scientists, pseudo-scientists, influencers, and politicians has become 
increasingly clear. The question of whom one can trust, whom one can believe, is becoming ever more common. 
The solutions to this problem need to be long-term and systematic and developed especially by means of: 


e developing critical thinking and critical decision-making of the whole society (or at least a large part 
of it, as true believers can never be convinced by the power of arguments), 

e — facts (logical argumentation, good science) versus pseudo-facts (bad science) arising from the problem 
of interpretation (System 1 or System 2), 

e —astrategic separation of truth from potential lies (unverified or poorly verified facts) or actual lies (de- 
ception and manipulation), which are a result of using pseudo-scientific methods and misleading use 
of various statistical explanations and shortcuts. 


It is entirely understandable that many are left confused and uneasy by the vocal assertions of fringe figures, 
but the onus of proof is always on those making grand claims. The history of science is littered with the hubris of 
the arrogant and misguided, and mere credentials are no impediment to being wrong; only evidence truly mat- 
ters. When confronted with the pronouncement of fringe figures, the motto of the Royal Society should always 
be at the forefront of our mind: Nullius in Verba (take nobody’s word for it) (Grimes, 2021). 


3 Motivated reasoning is a phenomenon studied in cognitive science and social psychology that uses emotionally biased reasoning to 
produce justifications or make decisions that are most desired rather than those that accurately reflect the evidence, while still reducing 
cognitive dissonance. 
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Abstract. There are limited research 
studies about the development of test 
instrument to assess the level of entrepre- 
neurial thinking among children in STEM 
education. The purpose of this research 
was to develop an Entrepreneurial Science 
Thinking Test (ESTT) for primary school 
children in STEM Education and evaluate 
its validity and reliability. The ESTT was de- 
veloped using experiential learning theory 
which comprised of five constructs, namely 
Observation, New Ideas, Innovation, 
Creativity, and Value. The ESTT consisted of 
ten open-ended question items that require 
children to answer questions in statements 
and draw sketches of ideas. The evaluation 
was conducted to determine the reliability 
and validity of ESTT which involved five 
subject matter experts and 166 11-year- 
old fifth graders from five urban schools in 
Sabah, Malaysia. The data obtained from 
fifth graders were computed using WIN- 
STEPS software version 3.73 and analysed 
using the Rasch measurement model. The 
results indicated a high acceptable content 
validity and construct validity, high internal 
consistency, and excellent item reliability 
and item separation. Through item fit 
analysis, all items were retained. The find- 
ing established the reliability and validity 
of the ESTT and would therefore represent 
a valid and highly reliable instrument for 
measuring entrepreneurial science thinking 
among fifth graders in STEM Education. 


Keywords: experiential learning theory, 
science entrepreneurial thinking, validity 
and reliability, STEM education 
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DEVELOPMENT OF A CHILDREN 
ENTREPRENEURIAL SCIENCE 
THINKING TEST FOR STEM 
EDUCATION 


Jamilah Ahmad, 
Nyet Moi Siew 


Introduction 


Entrepreneurial thinking (ET) is one of many critical skills that 21st 
century learners require in facing an increasingly competitive world. ET has 
the potential to enrich human capital in gaining innovative knowledge, 
new revolutionary ideas, and strong ethical standards (Bacigalupo et al., 
2016). ET can be defined as a cognitive situation that seeks creative and 
innovative ideas as well as opportunities (Krueger, 2005). ET is not an ability 
that needs to be mastered in order to become a mere entrepreneur, but 
it is a soft skill that is required to enhance human growth, meet the job 
market and improve productivity (Bacigalupo et al., 2016). In producing 
learners who can master the characteristics and ethics of entrepreneurship, 
handle existing resources well and face future challenges efficiently, this ET 
is very important (Edwards-Schachter et al., 2015). Learners can be more 
analytical, creative, and inventive thinkers, successful in communication, 
and ethical workers with these skills (Lekashvili, 2013). 

The importance of ET for supplying skilled workers who can com- 
municate different soft skills in Science, Technology, Engineering and 
Mathematics (STEM) fields has been acknowledged by some countries 
in their 21% century curricula (Borowczak, 2015). However, integrating ET 
is still not widespread in STEM education. Eltanahy et al. (2020) stressed 
the importance of applying ET in STEM learning to raise the awareness of 
STEM design among learners and inspire them to be more entrepreneurial. 
Learners studying STEM without ET skills are less likely to have the capabil- 
ity to determine the value of products derived from the socio-economic 
aspects of society. This is significant because learners are currently showing 
high trends (61.4%) towards STEM-based careers (Hatisaru, 2021). Swartz 
and McGuinness (2014) stressed that thinking skills need to be explicitly 
taught in curricula, while Gunawan & Shieh (2020) recommended that ET 
is integrated into any STEM curriculum to produce a significant effect on 
learning. 
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Realizing the need for entrepreneurial thinking in STEM, the importance of ET began to be examined by 
some Malaysian local scholars. Buang et al. (2009) have proposed a combination of science process skills and 
entrepreneurial thinking known as Entrepreneurial Science Thinking (EST). This concept integrates problem- 
solving ideas and entrepreneurial elements to train learners to search for thorough ideas in solving a problem 
creatively (Syukri et al., 2013b). However, there is little research on the development of test instruments to 
assess the level of entrepreneurial science thinking among primary school children in STEM education. It is 
therefore important to develop an ET test for STEM education to ensure that ET is nurtured in STEM education 
from primary school level (Liu & Zhi, 2010; Menzies, 2012; Sheehan et al., 2018). The Entrepreneurship Science 
Thinking Test (ESTT) was developed in line with this to assess entrepreneurial science thinking among fifth 
graders in STEM education. 


Research Problem 


Entrepreneurial science thinking begins with an important basic science skill which is the skill of creating 
(Daniel, 2016; Neck & Greene, 2011). In the concept of classroom assessment in Malaysian schools, learners 
with good creating skills can achieve the mastery level six (ML6). However, based on the Classroom Assess- 
ment Mastery Achievement Report (Tawau District Education Office, 2019), the number of tenth graders who 
achieved ML6 for Science subjects was only 4%, far behind when compared to ML5 (23.82%) and ML4 (44.25%). 

Furthermore, based on the PISA Report 2018 (Schleicher, 2019), the average score of Malaysian ninth 
graders in terms of science literacy is only 438, below the international average score of 489. This is a discon- 
certing scenario because the nation holds high aspirations to achieve a nation of creators by 2030. Therefore, 
it is obvious that entrepreneurial science thinking needs to be infused from the primary school level to better 
develop the children’s science literacy, critical thinking, and reasoning skills (Hoachlander & Yanofsky, 2011; 
National Research Council, 2011). 

The application of entrepreneurial science thinking in STEM Education is now considered significant in 
preparing children today to venture into STEM-related careers (Dabney et al., 2012; Sadler et al., 2012; Wyss et 
al., 2012). Such application will encourage children to solve challenging and meaningful daily problems as well 
as improve their cognitive reasoning skills (Hunter et al., 2016). In addition, entrepreneurial science thinking 
opens up space for learners to think more broadly to explore new ideas in STEM-based problems (English et 
al., 2017). Children with entrepreneurial science thinking are able to learn STEM in real context and develop 
their STEM literacy in order to succeed in the modern economic era (Tsupros et al., 2009). In fact, they are also 
able to face the challenges of daily life related to the field of STEM (Bybee, 2013). In addition, entrepreneurial 
science thinking also encourages children to improve Science literacy (McDonald, 2016) and engineering de- 
sign skills emphasized in STEM (Afriana et al., 2016; Jin & Bierma, 2013; Kennedy & Odell, 2014; Kuenzi, 2011; 
Zollman, 2012). It is clear that the application of entrepreneurial science thinking in STEM Education needs to 
be implemented explicitly starting at a lower school level to achieve the country’s desire to produce STEM- 
skilled human capital capable of solving global problems, making decisions, and innovating creations for the 
benefit of future societies. 

Although entrepreneurial thinking has been introduced in the Malaysian curriculum, the development of 
instruments that measure the level of entrepreneurial science thinking among primary school children in STEM 
education is not yet widespread (Buang et al., 2009; Syukri et al., 2013a). Instruments that have been developed 
previously only measure the readiness of the integration of entrepreneurial science thinking (Ishak et al., 2014) 
and teacher pedagogical knowledge in teaching entrepreneurial science thinking (Syukri et al., 2013b). 

Li et al. (2016) developed an instrument for measuring entrepreneurial thinking among engineering stu- 
dents. The instrument was developed based on Kern Entrepreneurial Engineering Network (KEEN) framework 
that is specialised in the context of engineering. The instrument focuses on three aspects, namely Curiosity, 
Connections, and Creating Value. The instrument reduced its items from 37 items to 29 items after the validity 
and reliability analysis using exploratory factor analysis. Unfortunately, the items do not reflect the development 
of new ideas and positive values in students’ creation. Bolton and Lane (2012) developed an Individual Entre- 
preneurial Orientation (IEO) instrument for testing entrepreneurial thinking among 1,100 university students. 
By using exploratory factor analysis, the instrument was found reliable and valid in assessing three of the five 
dimensions: innovativeness, risk-taking, and proactiveness. However, the measurements are comparatively 
restricted in these three dimensions. The extent to which the students’ observation of current materials and 
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designs, as well as the contribution of inventions to society, was not assessed. Schelfhout et al. (2016) con- 
structed an instrument to assess entrepreneurial competence using a behavioural indicator scale, in addition 
to measuring entrepreneurial thinking. The instrument was developed with 11 sub-competencies that assess 
high school students’ competence and entrepreneurial thinking. 

To sum up, most instruments constructed for measuring entrepreneurial thinking are appropriate for 
learners of higher education and secondary school. There are few and limited instruments targeting entrepre- 
neurial science thinking for primary school children in STEM education. Hence, an instrument for assessing 
entrepreneurial science thinking for STEM Education needs to be developed and introduced at a lower school 
level. In line with this, the Entrepreneurship Science Thinking Test (ESTT) was developed, and its validity and 
reliability were assessed to ensure its usability in measuring entrepreneurial science thinking among fifth 
graders in STEM education. 


Literature Review 
Science Entrepreneurial Thinking 


The concept of Entrepreneurial Science Thinking (EST) is a notion of science teaching and learning to pro- 
duce learners who are equipped with entrepreneurial thinking (Syukri et al., 2013a). EST is referred to as design 
thinking skills based on scientific knowledge and entrepreneurial orientation (Buang et al., 2009). Buang et 
al. (2009) proposed the Entrepreneurial Science Thinking Model (ESTM) which provided the theoretical model 
of entrepreneurial thinking in this study. ESTM consisted of five constructs namely Observation, New Ideas, 
Innovation, Creativity, and Value as described below. 


Observation 


The observation construct refers to the activity of observing various phenomena found in the environment 
that is related to the concept of science (Buang et al., 2009). The observation construct, as described in this 
study, is the activity of making observations in a planned and purposeful manner. Learners make observations 
about the appropriate materials and designs used to make a new product. 


New Ideas 


There is a need for new ideas through phenomena that have been observed in the form of ideas, systems, 
models, designs, or products (Buang et al., 2009). In this research, the new ideas construct suggests that learn- 
ers generate ideas by continuously thinking about the use of materials and the design of materials used in 
new products to seek uniqueness. 


Innovation 


Buang et al. (2009) stressed that the innovation construct refers to the activity of learners selecting some 
ideas that can be improved and evaluating the selected ideas. In this research, the innovation construct allows 
students to select ideas that can be modified or improved from the previous steps. In the future, the selec- 
tion of these ideas for change helps in creating product designs. Learners then need to evaluate the ideas by 
specifying the reasons for the idea they selected. 


Creativity 


The definition of creativity can be applied in a concentrated way as the task of creating new ideas (Buang 
et al., 2009). In this study, the creativity construct refers to the task of reinforcing and evolving ideas in a con- 
centrated way. Learners express their new ideas by sketching designs, labelling, and building product design 
models. Learners are then asked to state the name of the product, the price offered and the target group of 
buyers for the product produced. 
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Value constructs refer to activities to ensure that the ideas or products produced are beneficial to society. 
In terms of cost savings, product function as well as values and ethics in product invention, learners need to 
state the benefits of their products. This is to cultivate values such as love of the environment and the practice 
of using sustainable materials in the production of products for the community. Learners then present their 
model in front of the class and present the benefits of their products. 


Experiential Learning Theory in ESTT Development 


Kolb (1984)'s experiential learning theory defines learning as “the process whereby knowledge is created 
through the transformation of experience” (p. 41). Kolb proposed that learners learn through a circle of ex- 
periences that lead to observations and reflections. These reflections are applied to prior understanding and 
converted into abstract concepts or theories, resulting in forms and behaviours to adapt to new experiences 
that can be tested and explored. As claimed by Gemmell and Kolb (2013), experiential learning theory is in 
line with the concept of entrepreneurial science thinking which is based on learning through reflection and 
experience. Entrepreneurial science thinking needs to go through social and cognitive processes where learners 
develop new ideas and transform those ideas so that they can be used in the market and society (Gemmell et 
al., 2012). Thus, this theory provided the theoretical framework of ESTT development where learners use their 
experience to solve consumer problems and produce innovative products and services for the benefit of society. 

Kolb (1984) further reiterated that individuals progress through the four stages of the learning cycle that 
are interconnected, namely concrete experience, reflective observation, abstract conception, and active ex- 
perimentation (Figure 1). During the stage of concrete experience, learners are involved with new experiences 
which become the basis of observation. Next, in reflective observation, individuals reflect on their observation 
and begin to construct a general theory of what this information means. In the next step of abstract conceptu- 
alization, learners organize information systematically and logically into concepts, theories, and ideas. Finally, 
during the active experimentation stage, learners test the implications of these concepts in new situations. At 
this stage, learners are able to see things from various perspectives and act creatively and innovatively. The 
process once again returns to the first stage of the experience process. 

The learning process of Kolb (1984)’'s theory is in line with ESTT development, which encourages learners 
to experience the phases of Observation, New Ideas, Innovation, Creativity, and Values. Learners were exposed 
to issues through concrete experiences that would lead to the emergence of various ideas, which is in line 
with Observation in ESTT. This process leads to reflective thinking where learners obtain ideas and inspiration 
to invent products that could help in problem-solving. Hence, the reflective thinking stage is in line with New 
Ideas in ESTT. In the abstract conceptual stage, learners began to organize ideas and translate ideas in the 
form of prototype sketches. This is where innovation and creativity in ESTT take place. At this stage, through 
the experience they went through, learners invent their prototype. Carlsson et al. (1976) explained that the 
process of implementing solutions to issues would be carried out during the stage of active experimentation. 
Accordingly, learners evaluated their products by ensuring that the products invented could solve issues and 
benefit the users. As explained by Krakauer et al. (2017), the stage of active experimentation is the stage where 
learners relate their perceptions to the actual context. In this study, the actual context refers to the context of 
society where learners applied their STEM knowledge and experience in inventing the products that would 
benefit society. This stage represents the Value construct of ESTT. 
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Figure 1 
Learning cycle in Kolb’s Experiential Learning Theory (1984) 
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Abstract 
Conceptualization 


Rasch Measurement Model 


The Rasch Measurement Model (RMM) is an efficient solution for the development of in-depth statistics to 
provide high validity and reliability of ESTT instrument (Bond & Fox, 2015). RMM analyses each respondent's ability 
to respond to the instrument and measures the difficulty of each item in the instrument (Wolins et al., 1982). RMM 
is also able to assess latent characteristics, such as human thoughts and emotions (Aziz et al., 2015). 

The RMM built on the basis of Item Response Theory is among the adequate statistical models as it can cal- 
culate the item’s difficulty and the individual's ability to be evaluated at the same time (Deane et al., 2016). As a 
result, the RMM was able to classify the items and person validity and reliability. Furthermore, Rasch analysis can be 
used in terms of item polarity, item and respondent misfit, as well as dimensionality, to perform construct validity. 

While Rasch analysis can take a longer process than conventional analysis, Rasch analysis may offer a deeper 
understanding of the instrument's strengths and shortcomings (Boone & Scantlebury, 2005). Bond and Fox (2007) 
claimed that RMM, by statistical analysis, is an efficient solution for developing highly valid and reliable instru- 
ments. The researchers used Rasch analysis to assess the validity and reliability of the ESTT instrument based on 
these specified strengths. 


Research Focus 
The main focus of this research was to develop an instrument on entrepreneurial science thinking in STEM 
Education for 11-year-old children in primary schools. Validity and reliability are two important factors to consider 
when developing an instrument. Hence for the need to develop a valid and reliable tool, Rasch analysis was used 
to examine the validity and reliability of ESTT. 
Research Aim and Questions 
This research was aimed to develop a valid and reliable instrument for assessing entrepreneurial science 


thinking in STEM Education among fifth graders using the Rasch Measurement Model. There were three research 
questions guiding this research: 
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Q1: IsESTT feasible in assessing entrepreneurial science thinking among fifth graders in STEM Educa- 
tion? 

Q2: How valid is ESTT in assessing fifth graders’ science entrepreneurial thinking in STEM Education 
based on: 1) Person fit statistics; 2) Item fit statistics; 3) ltem Polarity; and 4) Confirmation of uni- 
dimensionality? 

Q3: Howreliable is ESTT in assessing the entrepreneurial science thinking in STEM Education among 
fifth graders according to: 1) KR-20 Cronbach's coefficient alpha; 2) Person and item reliability 
indices? 


Research Methodology 
Research Design 


A survey design using a test instrument was employed to obtain data on fifth graders’ entrepreneurship 
thinking level. In education, a survey research design is well established as it can analyse topics and constructs 
efficiently and economically (Creswell & Creswell, 2017). The data gathered from the ESTT instrument was focused 
on the fifth graders’ experiences of integrating the entrepreneurship elements in science project-based learning. 
This research was carried out between November and December 2019. 


Research Sample 


The ESTT was distributed to 166 samples that were chosen randomly from five primary schools in the Tawau 
district, Sabah, Malaysia. A sample size with the range of 108-243 is wide enough to have 99% confidence that 
the item difficulty can be measured within + 1/2 logit of its stable value (Linacre, 1994). The research sample had a 
similar background where the selected schools were grouped in urban school clusters. The sample consisted of 87 
girls (52.4%) and 79 boys (47.6%) aged between 10 to 11 years. Approximately 60% of parents were government 
employees, while 40% were working in the private and industry sectors. 


Ethical Considerations 


Written consent was obtained from the parents and school principals prior to the administration of ESTT. 
Initially, an outline of the intent of the ESTT which was to measure the level of entrepreneurial science thinking 
was meted out. The consent letter detailed the fifth graders’ involvement in the research and the parents’ consent 
reflecting their understanding of the research purpose. All the fifth graders were assured of the confidentiality of 
their responses and complete anonymity. Fifth graders were also informed that anyone could withdraw from the 
research without penalty. A follow-up briefing session was then carried out to explain the guidelines and proce- 
dures for answering questions in ESTT. 


Administration of ESTT 


Each fifth grader was asked to read the instructions carefully before answering the ESTT according to their respec- 
tive knowledge without the help of other children. Fifth graders relate current information, insights, and experiences 
to the scientific principles and entrepreneurial aspects surrounding them throughout answering questions, such as 
building materials, the use of construction materials, design, uniqueness of an invention, and the impact of an inven- 
tion on society. Fifth graders were given an hour to think and articulate their responses. Open questions were given 
according to the construct and fifth graders were required to answer the questions in the order of the constructs. The 
answers were collected and reviewed in advance to ensure the fifth graders followed the correct instructions and 
provided complete answers before the data are analysed using WINSTEPS software version 3.73. 


Instrumentation 


Entrepreneurial science thinking is the ability to create and make improvements to a product, idea, or process 
so that the product has added value from the social and economic aspects. In this regard, the relationship of sci- 
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ence concepts acquired in the classroom and the ability of teachers in nurturing EST triggers innovation to adapt 
to situations that occur and are needed in the daily life of children (Venuvinod & Sun, 2002). This concept was used 
when developing the question items in the Entrepreneurial Science Thinking Test (ESTT). 

Open-ended questions were used in measuring the entrepreneurial science thinking of each individual. 
The use of open-ended questions can help in obtaining variations in respondents ideas (Chen et al., 2020), provide 
an overview of respondents’ level of knowledge (Clarke & Holt, 2019), and help researchers identify misconceptions 
among respondents (Schuetz, 2010). Researchers constructed open-ended questions based on the five constructs 
in EST (Buang et al., 2009) which are Observation, New Ideas, Innovation, Creativity, and Value. 

The ESTT consisted of ten open-ended question items that require fifth graders to answer questions in state- 
ments and draw sketches of ideas. Question items were developed by referring to the content of Standard Docu- 
ments of Curriculum and Assessment (DSKP) for Year Five Science under the themes of Physical Science as well as 
Technology and Sustainable Life (Curriculum Development Division, 2019, p. 55-68). The main question asked fifth 
graders to produce cell phone designs for the use of the future community. The context of cell phone usage was 
chosen because it is one of the criteria in DSKP under the theme of Physical Science. Fifth graders were then given 
a stimulus picture and ten question items. Items were arranged according to the group of constructs to enable 
fifth graders to organize their answers which then lead them to Entrepreneurial Science Thinking. Table 1 shows 
the constructs of the Entrepreneurial Science Thinking Test (ESTT) for STEM Education. 


Table 1 
Items for ESTT by Construct 


Construct Definition Item No. Item statement 
Observation Make observations in a planned la State the materials used to make cell phones. 
and purposeful manner. 1b State the design of a cell phone that you can observe. 
New Ideas Generate ideas by looking for 2a Explain the advantages of using the materials you mentioned to build the 
uniqueness cell phone. 
2b Explain the advantages of using the cell phone design you mentioned. 
Innovation Select ideas that can be im- 3a Based on the ideas you have stated in question 2, select three (3) ideas 
proved and evaluate those ideas that you can improve to produce future cell phones. 
3b Why did you choose the above ideas? Give three (3) reasons. 
Creativity Strengthen and improve ideas in Reinforce and improve the three (3) ideas you have chosen for the crea- 
a focused way 4a tion of new cell phones for future community use. Sketch and label new 
features on your model in the space provided. 
4b Complete details about your product by stating the product name, price 
offered and target group of buyers. 
Value Ensure that the ideas or prod- 5a State the benefits of your product to the community in terms of cost sav- 


ucts produced are beneficial to ings and product functionality. 
the community 


State the benefits of your product to the community in terms of values and 


0 ethics in product creation. 


Children were required to make observations in a prepared and purposeful way in the Observation Construct. 
The first item required the children to state the materials used to make a cell phone, whilst the second item required 
the children to state the cell phone design. Children were required to produce ideas relevant to individuality in 
the creation of New Ideas. By specifying the benefits of utilising the products and design requirements in the 
observation stage, children had to search for uniqueness or benefits. The fifth item in the Innovation Construct 
instructed children to choose three ideas that could be augmented and updated in order to create potential cell 
phones, while the sixth item requested children to state the reasons for choosing the ideas mentioned. The Creativ- 
ity Construct allows children to validate and strengthen concepts in a focused way. The seventh item centred on 
children strengthening and improving the concepts chosen for the development of new cell phones. They were 
requested in this item to draw and mark new features on the model they made. Next by specifying the name of the 
product, the price available, and the target category of purchasers, the eighth item advised children to complete 
the details of the product they had made. Finally, the Value Construct was where children wanted to ensure that 
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the products would help the society. The ninth and tenth items allowed children to state the advantages in terms 
of cost savings and include terms of values and ethics in product development, respectively. 

The time allocation for answering all the items was 60 minutes. The scoring rubric for the Entrepreneurial Sci- 
ence Thinking Test construct was adapted from Ho et al. (2013). There is a minimum score of 0 and a maximum score 
of three for each item given in the test. Based on the child's response, each score was rated from 0 to 3 points. An 
example of a scoring rubric according to children’s response for one of the constructs in ESTT is shown in Table 2. 


Table 2 
An Example of the Scoring Rubric according to Children’ Response in ESTT 


Construct Ability Scoring rubric pevelealie 

Scores 

3. Innovation 3.2 Make an evaluation of the ideas 3.2.4 Children can provide three (3) factors in the Level 4 

(Select ideas that can be that have been selected. selection of ideas. 3 marks 
improved and evaluate ; ; ; 

those ideas) 3.2.3 Children can give two (2) factors in the selec- Level 3 

tion of ideas. 2 marks 

3.2.2 Children can provide one (1) factor in the Level 2 

selection of ideas. 1 mark 

3.2.1 Children cannot give one (1) factor in the Level 1 

selection of ideas. 0 mark 

pe Gneose eee) a he a 3.1.4 Children can choose three (3) ideas peel 

proved to produce new inventions. 3 marks 

3.1.3 Children can choose two (2) ideas pies 

2 marks 

F ‘ : Level 2 
3.1.2 Children can choose one (1) idea 

1 mark 

3.1.1 Children cannot choose one (1) idea Fev! 3 

0 mark 


Source: Ho et al. (2013) 


Data Analysis 


The data of the research were analysed to determine the content and construct validity. To determine the value 
of content validity agreement, the researchers used the Content Validation Index (CVI). CVI provides an average 
rating of scores for all items evaluated by an expert. A CVI value can be computed for each item on a scale (I-CVI) 
as well as for the overall scale (S-CVI). Davis (1992) pointed out that for a newly designed instrument, the usual CVI 
value obtained is .80. For cases with content validation requiring three or more experts, Polit et al. (2017) recom- 
mended a rating of .78 and above for I-CVI and a minimum S-CVI of .80 for the averaging approach. 


Item Content Validation Index (I-CVI) = Total of experts in agreement 
Total of experts 


Scale Content Validation Index/Average (S-CVI/Ave) = Total I-CVI for each item on the scale 
Total of items 


For assessing construct validity and item reliability, WINSTEPS software version 3.73 was used. This en- 
sured the instrument's quality and the accuracy of the data obtained by the researcher before the instrument was 
used in actual research. The first analysis was conducted via a person fit analysis which was based on the values 
of ‘MEASURE; MNSQ Outfit and ZSTD Outfit (Edwards & Alcock, 2010). Nevin et al. (2015) asserted that if the ZSTD 
Outfit value exceeds 2.0 and the MEASURE value is high, there is a probability that excellent learners would not 
carefully answer the easy items. If the ZSTD Outfit value exceeds 2.0 but the MEASURE value is low, it is likely that 
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low-ability learners would be able to answer difficult items correctly. Therefore, unfit respondents will be eliminated 
to increase the validity of the instrument (Lamoureux et al. 2008). 

For the item fit analysis, Boone et al. (2014) as well as Bond and Fox (2015) proposed three criteria namely, Outfit 
Mean Square Values (MNSQ), Outfit Z-Standardised Values (ZSTD) and Point Measure Correlation (PTMEA-CORR). 
The MNSQ Outfit value informs the researcher about the item fit in the measurement while the PTMEA-CORR value 
indicates whether the development of the construct has achieved its goal (Bond & Fox, 2007). In another aspect, 
ZSTD provides information to the researcher whether the data obtained really conforms to the instrument model. 
Any item that fails to meet one of the criteria in Table 3 needs to be modified or dropped so that the item fit value 
can be increased (Sumintono & Widhiarso, 2015). 


Table 3 
Fit Indices for Item Fit 
Statistics Fit Indices 
Outfit mean square values (MNSQ) 0.50 - 1.50 
Outfit z-standardised values (ZSTD) -2.00 - 2.00 
Point Measure Correlation (PTMEA-CORR) 0.40 - 0.85 


Source: Boone et al. (2014) 


Rasch analysis can also be used to identify item polarity through PTMEA-CORR values. A positive PTMEA- 
CORR value indicates that the item can accurately measure what it needs to measure and vice versa if its value 
is negative. The researchers also assess the instruments’ unidimensionality to guarantee that the instrument can 
successfully measure the construct of entrepreneurial science thinking (Sumintono & Widhiarso, 2015). Component 
Analysis provides dimensional criteria based on the ‘raw variance explained by measures’ (Sumintono & Widhiarso, 
2015). The accepted value of ‘raw variance explained by measures’ should exceed 20%, is considered good if it is 
more than 40% and considered excellent if more than 60%. Meanwhile, the value of ‘unexplained variance in first 
contrast’ should not exceed 15%. 

In terms of reliability, the researcher referred to Sumintono and Widhiarso (2015) for Cronbach's alpha value 
(KR-20), item-person reliability and separation indices (Table 4). The separation index of persons was used to classify 
the level of learners. A good separation index should be >2, where the higher the separation index, the better the 
level of classification of persons. Item separation index was also used to validate the item hierarchy. The low item 
separation index value, <3 shows that the sample of learners was not large enough to confirm the hierarchy of 
item difficulty in the instrument. Linacre (2002) insisted that a high separation value indicates that the instrument 
has good quality since it can identify the group of items and respondents. 


Table 4 
Reliability Measured via the Rasch Analysis 


Statistics Fit Indices Interpretation 
Cronbach's alpha (KR-20) <5 Low 
<6 Moderate 
6-7 Good 
7-8 High 
9-1.0 Very High 
Item and Person Reliability Index <.67 Low 
.67 - .80 Sufficient 
81 -.90 Good 
91 - .94 Very Good 
>.94 Excellent 
Item Separation Index >3 Good 
Person Separation Index >2 Good 


Source: Sumintono & Widhiarso (2015) and Linacre (2002) 
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Research Results 
Content Validity 


Content validity indicates the extent to which an item adequately represents the content of a trait that a 
researcher wants to measure (Creswell & Creswell, 2017). Kline (2005) stated that an experts’ review is necessary 
to ensure the accuracy of the construct as well as the clarity of its contents. Mullen (2003) remarked that a group 
of experts are those who are trained in a specific field. Therefore, the researchers showed the ESTT to five experts 
who had vast knowledge in the fields of entrepreneurial science thinking, Science Education and STEM Education. 
The researchers used an item evaluation form adapted from the Malaysian Examinations Board (2013). The panel of 
experts evaluated ESTT items from the aspects of conformity, accuracy, and clarity as well as suitability. Comments 
from the experts were recorded and taken into consideration for the ESTT instrument improvement process. Table 
5 shows the list of experts involved in the content validation panel. 


Table 5 
Content Validation Panel for ESTT 


Name Institute Designation Expertise 

Expert A University Professor (PhD) Science Entrepreneurial Thinking 
Teachers Training Academic Lecturer : . ; 

Expert B Institute (TT!) STEM Department (PhD) Curriculum and Instructional (Science) 
Teachers Training Academic Lecturer : ; 

Epere Institute (TTI) STEM Department Suances Education 

Expert D Primary School ear mare Pane STEM Education and Scientific Creativity 

Expert E Primary School Science Subject Coach Science Education (Primary School) 


Based on Table 6, all items in ESTT were accepted and the value obtained from the Scale Content Validity 
Index (S-CVI) was .92. This S-CVI value meets the requirement of = .80 as set by Polit et al. (2017) for new instru- 
ments. The S-CVI index of .92 indicated that the expert panel regarded the content validity of the ESTT instrument 
as very high and acceptable. 


Table 6 
Content Validity Index (CVI) Results of the ESTT Instrument 


Experts in 


Item ExpertA ExpertB  ExpertC ExpertD ExpertE Agreement I-CVI Results 
la / / / / 4 80 Accepted 
1b / / / / 4 80 Accepted 
2a / / / / 4 80 Accepted 
2b | / / / 4 80 Accepted 
3a / / / / / 5 1.00 Accepted 
3b / / / / / 5 1.00 Accepted 
4a / / / / / 5 1.00 Accepted 
4b / / / / / 5 1.00 Accepted 
5a / / / / / 5 1.00 Accepted 
5b / / / / / 5 1.00 Accepted 

Scale Content Validation Index/ Average (S-CVI/Ave) 92 
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Table 7 displays the persons (which are the fifth graders in this case), whose responses were most ill-fitting with 
the Rasch analysis; or in other words, their responses contrasted from the estimation given by the Rasch model. The 
fifth graders were ranked according to the highest ZSTD Outfit value. Among the fifth graders, there were seven 
respondents (063, 100, 127, 166, 004, 002 and 022) who scored an Outfit ZSTD value higher than 2.0. The remain- 
ing fifth graders scored an Outfit ZSTD value within the acceptable range (from -2.0 to +2.0). Fifth graders whose 
responses gave negative PTMEA-CORR readings showed that they made decisions out of the usual. 


Table 7 
Misfit Order of the Persons in ESTT Instrument 


Person MNSQ Outfit ZSTD Outfit PTMEA-CORR 
(.50-1.50) (-2.0-2.0) (.40 - .85) 
063 9.90 3.9 - 38 
100 9.90 4.5 - 44 
127 9.90 3.1 - 25 
166 7.33 2.5 - 12 
004 5.72 2.8 - 02 
002 2.97 2.2 08 
022 2.97 2.2 .08 


In addition, there were 10 fifth graders who attained extreme marks (maximum marks) namely, students 
020, 044, 068, 076, 084, 088, 093, 096, 109, and 161 (Figure 2). A total of 17 fifth graders were excluded and 149 
respondents out of the 166 respondents were involved with the next analysis. This denotes that in the pilot study, 
the items were fit for almost all the students (89.76%) and the analysis conducted on those fifth graders showed 
quality findings for the assessment using the Rasch analysis. 


Figure 2 
Person with Extreme Scores 


INPUT: 166 Person 10 Item REPORTED: 166 Person 10 Item 3 CATS WINSTEPS 3.73 


Person: REAL SEP.: 2.44 REL.: .86 ... Item: REAL SEP.: 6.86 REL.: .98 


Person STATISTICS: MEASURE ORDER 


JENTRY TOTAL TOTAL MODEL| INFIT | OUTFIT |PT-MEASURE |EXACT MATCH| | 
|NUMBER SCORE COUNT MEASURE §S.E. |MNSQ ZSTD|MNSQ ZSTD|CORR. EXP.| OBS% EXP%| Person] 
------------------------------------ +--- ---------+-----------+------- 
| 20 30 10 7.51 iT MAXIMUM MEASURE| 9.00 .00/100.0 100.0/ 020 | 
| 44 30 10 7.51 1 MAXIMUM MEASURE| J} .00 .00/100.0 100.0| 044 | 
| 68 30 10 7.54 1 MAXIMUM MEASURE| § .00 .00/100.0 100.0] 068 | 
| 76 30 10 7.51 4 MAXIMUM MEASURE|}.00 .00/100.0 100.0| 076 | 
| 84 30 10 7.51 1 MAXIMUM MEASURE| 9} .00 .00/100.0 100.0| 084 | 
| 88 30 10 4:54 1 MAXIMUM MEASURE| § .00 .00/100.0 100.0] 088 | 
| 93 30 10 7-51 cf MAXIMUM MEASURE|f}.00 .00/100.0 100.0/ 093 | 
| 96 30 10 7.34. 1 MAXIMUM MEASURE] § .00 .00/100.0 100.0] 096 | 
| 109 30 10 7.51 = MAXIMUM MEASURE| § .00 .00/100.0 100.0] 119 | 
| 161 30 10 7.51 1 MAXIMUM -00 .00/100.0 100.0| 161 | 
| 1 29 10 6.13 1 ares: . 22 29| 90.0 90.0] 001 | 
| 3 29 10 6.13 : -4| BE i | 24 29} 90.0 90.0] 003 | 
| 12 29 10 6.13 1 = 51 -31) 222 29| 90.0 90.0] 012 | 
| 16 29 10 6.13 iS r -4| oe | 28 29| 90.0 90.0] 016 | 
I 19 29 10 6.13 1.1211.18 -51: =%9 .31 22 29! 90.0 90.0! 019 | 
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Item Fit 


Linacre (2007) emphasised that the value of MNSQ Outfit provides a strong value in determining the suit- 
ability of items for the measurement of a construct. Boone et al. (2014) ascertained that the suitable product value 
range is between .5 to 1.5 based on MNSQ Outfit, between -2 to +2 range for ZSTD Outfit and between .4 to .85 
range for PTMEA-CORR. Any items that are outside the range of Outfit MNSQ, Outfit ZSTD, and PTMEA-CORR are 
considered inappropriate (Boone et al., 2014). However, if the item meets one of the criteria, the item must be 
retained (Sumintono & Widhiarso, 2015). 

Based on Table 8, there are four items (1, 4, 7 and 9) that are outside the range of items. Boone et al. (2014) 
and Aziz et al. (2014) stated that items situated outside the range and which do not meet all three criteria are 
considered unsuitable. However, if the item meets one of the criteria, the item must be retained (Sumintono & 
Widhiarso, 2015). Table 8 shows that all the items meet at least one criterion. Thus, no items were changed and 
removed from the instrument. 


Table 8 
Misfit Order of the Items in ESTT 


Outfit MNSQ Outfit ZSTD PTMEA-CORR 


en (.50-1.50) (-2.0 - 2.0) (.40 - .85) nea 
I7 1.57 3.2 54 retained 
12 1.35 a 18 retained 
15 1.33 9 19 retained 
13 1.26 1.2 CL retained 
18 55 -4 16 retained 
19 36 -.9 69 retained 
16 92 -5 64 retained 
110 87 -1.0 85 retained 
14 67 -27 84 retained 
1 52 3.5 .90 retained 


Item Polarity 


Based on Table 8, the minimum value of PTMEA-CORR is .54 while the maximum value is .90. These positive 
values indicated that all the items functioned in a parallel direction while negative values showed which items 
needed to be repaired or dropped. The item polarity analysis using PTMEA-CORR values elucidates that the items 
in ESTT move in the same direction according to the measured construct (Linacre, 2002). 


Unidimensionality 


Unidimensionality is important to determine that the instrument being developed can measure in one direction 
and ensure precise results from the study. Unidimensionality is frequently detected by using the Principal Compo- 
nent Analysis of Rasch Residual (PCAR). Based on Figure 3, the observed value of the Raw Variance Explained by 
Measures is 66.8%. Values exceeding the 60% index are at an excellent level and it is proven that ESTT instruments 
have a strong unidimensionality. This shows that ESTT can measure constructs in science entrepreneurial think- 
ing. The value of 6.6% was detected from the Unexplained Variance in the 1% Contrast, which was less than 15%. 


539 


https://doi.org/10.33225/jbse/21.20.528 LIT 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


DEVELOPMENT OF A GHILDREN ENTREPRENEURIAL SCIENGE THINKING TEST FOR STEM ISSN 1648-3898 print 
EDUCATION 

(PP. 528-545) ISSN 2538-7138 /Online/ 
Figure 3 


Principal Component Analysis of Rasch Residual (PCAR) 


INPUT: 149 Person 10 Item REPORTED: 149 Person 10 Item 3 CATS WINSTEPS 3.73 


Table of STANDARDIZED RESIDUAL variance (in Eigenvalue units) 

-- Empirical -- Modeled 

30.1 100. 0% 100. 0% 

20.1 66. 8% 65.7% 
5 . OF 44.9% 


Total raw variance in observations 
Raw variance explained by measures 
Raw Variance explained by persons 


Raw Variance explained by items = 6.4 21.2% 20. 8% 
Raw unexplained variance (total) = 10.0 % 100.0% 34.3% 
UnexpIned variance in 1st contrast = 2.0 19.7% 
UnexpIned variance in 2nd contrast = 1-5 ~0% 15.0% 
UnexpIned variance in 3rd contrast = 1.2 4.0% 12.1% 
UnexpIned variance in 4th contrast = 1.2 3.8% 11.6% 
UnexpIned variance in 5th contrast = 1.1 3.7% 11.0% 


Reliability and Separation Indices 


In relevance to the reliability of ESTT, the person measure reliability indicates how well the fifth graders can 
be distinguished based on their responses. The item measure reliability specifies how well the items (statements) 
can be discriminated from one another based on their authenticity approved by the respondents. The reliability 
findings overview is summarized in Table 9. 

Table 9 presents the ESTT’s reliability values based on Cronbach's coefficient alpha (KR-20). The value achieved 
was .91. This infers that the reliability of the ESTT is within a very high range (Sumintono & Widhiarso, 2015). The 
item reliability value of .98 is within a very high range (Bond & Fox, 2007; Linacre, 2007). The item separation index 
was 7.97, which is more than 3.0. This value proved that ESTT has a good spread of items (Linacre, 2007) that it 
could distinguish the respondents according to the hierarchy of item difficulty in ESTT. 

The person reliability of .89 was also within a good range, whereas the person separation value in ESTT was 
2.83. Bond and Fox (2007) stated that the respondents reliability value, which is higher than .80, indicates that the 
respondents gave good and consistent feedback. The item separation value of more than 2.00 indicates there is 
enough spread of items (Linacre, 2002). The value of 2.83 points out that the fifth graders could be divided into 
three groups according to their responses to the items in ESTT. 


Table 9 
Summary of the Reliability Findings 


Rasch 


Hieaeurenaitt ESTT Interpretation 
Cronbach's alpha (KR-20) 9-1.0 91 Very high 
Item Reliability >.94 98 Excellent 
Item Separation Index > 3.0 7.97 Good 
Person Reliability 81 - .90 89 Good 
Person Separation Index > 2.0 2.83 Good 


Discussion 


In this research, ESTT was developed to assess the level of entrepreneurial science thinking among fifth grad- 
ers. In addition to its importance in determining the extent of entrepreneurial science thinking, this ESTT also offers 
added value in the analysis of entrepreneurial science thinking in STEM Education. 

Overall, the Rasch Measurement Model was used to assess the validity and reliability of ESTT. Five constructs, 
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namely Observation, New Ideas, Innovation, Creativity and Value, were established as the focus of the ESTT in- 
strument. All ten open-ended items went through the content validation phase by five experts. The items for the 
Observation Construct were adjusted from observation about energy and the transformation of energy to the 
observation of building materials and design. This is to guarantee that the scientific principles of materials are 
tested during the Observation Construct. In addition, the researcher also changed the question's keyword from 
‘what’ to ‘explain’ for the construction of the New Ideas construct. It is to ensure that fifth graders are subjected to 
the engagement of high-level thinking. 

The construct validity analysis was conducted for item and person fit, item polarity as well as unidimensional- 
ity. The results of Rasch’s analysis show that ESTT has good psychometric quality. A positive PTMEA-CORR analysis 
indicates that all items move in the same direction in measuring the constructs to be measured (Bond & Fox, 2015; 
Linacre, 2012). Meanwhile, the obtained Raw Variance Explained by Measures value of the ESTT instrument proved 
that the ESTT instrument truly measures the construct of entrepreneurial science thinking. In other words, there 
was no sixth construct in ESTT (Aziz et al., 2015; Fisher, 2007). 

The ESTT instrument was also tested for its reliability and separation index. According to Cohen and Swedlik 
(2018), a good set of test items can be distinguished by respondents. The ESTT instrument was found to have very 
high Cronbach's alpha value, excellent item reliability value and good person reliability. These findings show that 
the reliability of ESTT instrument is high in assessing the entrepreneurial science thinking of fifth graders in STEM 
education. The obtained good item separation value shows that ESTT instrument can be categorized into eight 
strata of items levels while the obtained person separation value proves that fifth graders can be divided into three 
strata according to ability level. 

Most instruments developed by previous researchers (Li et al., 2016; Bolton & Lane, 2012; Schelfhout et al., 
2016) concentrate on high school and undergraduate students’ level of entrepreneurial thinking in the field of 
engineering and entrepreneurship education. This limitation was addressed with the development of ESTT. 

Research findings confirm that ESTT is explicitly in line with Kolb (1984)’s experiential learning theory and 
offer a new instrument for primary school children in STEM education. ESTT helps educators obtain preliminary 
data on the extent of children’s entrepreneurial science thinking in primary schools. The data can then be used to 
develop appropriate education programs to enhance entrepreneurial science thinking. As reported by Gray (2016), 
the level of entrepreneurial thinking is essential to anticipate learners’ future potential in STEM. 

This study has also shown that the level of entrepreneurial thinking among primary school children can be 
assessed by referring to the appropriate entrepreneurial Thinking Model. Moreover, ESTT utilises an open-ended 
questions approach, which allows children to offer their views more freely based on their current experience and 
knowledge (Lifan & Chen, 2009). In addition to that, most previous entrepreneurial thinking instruments used the 
exploratory factor analysis method, while ESTT uses Rasch analysis, which leads to the accuracy of the instrument 
developed. 


Conclusions and Implications 


This research has several implications that impact practicality and methodology. In terms of practicality, the 
ESTT is a new instrument developed by researchers based on Kolb’s Experiential Learning Theory and the Entrepre- 
neurial Science Thinking Model and was adapted to suit the context of STEM Education. Thus, the Entrepreneurial 
Science Thinking Test (ESTT) instrument is fit to fill the gap in STEM education research and overcome the issue of 
the lack of instruments that measure entrepreneurial science thinking among primary school children. The develop- 
ment of this instrument is a significant endeavour in warranting the continuity of the application of entrepreneurial 
science thinking in STEM Education which will ultimately contribute to a generation of inventors and creators. 

In terms of methodology, the Rasch Model of Measurement is more specific and detailed in determining the 
validity and reliability of the ESTT instrument. The Rasch measurement model's analysis of instrument validity in 
terms of item-person fit, item polarity and unidimensionality showed that ESTT instrument has a high validity in 
measuring entrepreneurial science thinking in STEM education. The high reliability analysis of ESTT instrument 
with good item-person separation value proved that ESTT instrument is reliable for measuring entrepreneurial 
science thinking in STEM education for fifth graders. This study also demonstrated that instruments that have 
gone through a phase of validity and reliability will help the researcher measure the variables studied and make 
accurate decisions based on the analysis of the findings. In fact, these findings provide a basis for other research- 
ers so that the ESTT instrument can be distributed to learners in other areas. Ultimately, the validity and reliability 
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analysis using the Rasch Measurement Model successfully proved that the ESTT instrument is very suitable for real 
field study in assessing entrepreneurial science thinking in STEM education among children in primary schools. 

Even though the findings suggest that ESTT is a reliable and valid instrument for STEM education, its limita- 
tions should be acknowledged. ESTT was tested in five primary urban schools using a sample of 166 fifth graders, it 
may not be representative of the general population of primary school students. Future research needs to involve 
a bigger sample size, including rural schools. ESTT can be infused into any STEM curriculum integrated model to 
promote better critical and inventive thinking skills in STEM. It is suggested to extend the usage of ESTT instrument 
to other regions of the country, as well as to a variety of learners for greater generalisability. 
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Introduction 


The science curriculum is considered one of the pillars of scientific and 
technological progress; it contributes to developing learners’ knowledge and 
applied skills as well as expanding on science trends. However, there is a gap 
between what is taught in schools and what students need in their lives and 
to succeed in the job market. One means of addressing this gap in learning 
is taking advantage of global visions and experiences in science education. 

Because of the natural overlap between science, technology, engi- 
neering, and mathematics, science curricula have evolved to reflect the 
relationships among the fields. In the 1990s in the United States, the National 
Science Foundation created a consolidated science, technology, engineering 
and mathematics (STEM) education topic to expand the skilled and innova- 
tive workforce by enhancing students’ ability to build and produce science 
knowledge across the four integrated fields (Misher, 2014). 

Acommittee that consisted of the National Academy of Engineering and 
the National Research Council (NAE & NRC, 2014) defined science as studying 
and investigating the natural worlds, mathematics as studying relationships 
between quantities to build logical arguments, engineering as knowledge 
about design and product construction, and technology as knowledge, 
processes, and tools that relate to employing and producing technology. The 
mechanisms of integrating these fields vary because of differing educational 
strategies and different special features across fields (NAE & NRC, 2014). For 
instance, some curricula might aim to teach science, technology, engineering, 
and mathematics integrally, while others might incorporate some aspects of 
the STEM fields to support teaching other fields (Bybee, 2013). 

However, what best facilitates the integration of the STEM fields into 
teaching is focusing on major ideas, for example, connecting a problem or a 
concept to learners’ realities (Mcgehee, 2015). The Next Generation Science 
Standards (NGSS) were developed to reflect this enhanced understanding 
of science and engineering practices so as to build on learners’ knowledge 
through a variety of practices. Specifically, the new standards aimed to clarify 
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how to accomplish the following: define problems and ask questions, develop and use models, plan and conduct 
investigations, analyze and interpret data, use mathematics and computational thinking, interpret findings and 
design solutions, and generally participate in science debate based on obtaining, analyzing, interpreting, and 
communicating evidence (NRC, 2012). 

Several researchers have identified positive impacts of teaching in accordance with STEM education on forming 
positive attitudes toward STEM (Laforce et al., 2017; Lou et al., 2010; Misher, 2014), improving academic achieve- 
ment (Al-Qathami, 2017; Al-Shehimiah, 2015; Najjar, 2015; Wahono et al., 2020), and developing 21st-century skills 
(Husin et al., 2016), thinking habits (Al-Daoud, 2017; Najjar, 2015), thinking skills (Al-Qathami, 2017), decision-making 
skills (Al-Daoud, 2017; Rizk, 2015), creative thinking (Al-Shehimiah, 2015), and motivation (Wahono et al., 2020). 
Given the importance of STEM education and its reliance on teachers’ understanding and attitudes toward it (NAE 
& NRC, 2014), recognizing teachers’ perceptions of STEM education could contribute to helping them organize 
their knowledge and understand their behaviors (Owens, 2014; Turner, 2013). 

The global interest in STEM education is evident as it is included in 798 articles in 36 journals during the 
period of 2000 to 2018. However, the topics related to teacher and STEM teaching of K-12 comprised 12.9% (Li et 
al., 2020), and this may highlight the need for more research on teachers, particularly their perceptions, of what 
notions contribute to guiding teaching decisions (Shahzad et al., 2017). Some local and international studies that 
were conducted on science teachers’ perceptions of STEM education have shown positive perceptions of the STEM 
concept (Al-Anzi and Al-Jabr, 2017; Ambosaidi et al., 2015; Smith et al., 2015), and its importance (Amir et al., 2015; 
Park, et al., 2016; Turner, 2013; Wang, 2012). These results were revealed after teachers had received training courses 
on STEM education for 8 hours, as the study of Amir et al. (2015), or 3 weeks, as the Turner's (2013) study, or a year 
and a half, as Wang's (2012) study. On the other hand, the studies of Al-Anzi and Al-Jabr (2017) and Ambosaidi et 
al. (2015) revealed the perceptions of Science educators about the concept of STEM education and its teaching 
requirements prior to the enrollment in professional development programs about STEM. 

Several studies have revealed the relationship between teachers’ perceptions of STEM education and some 
variables. The first variable is teaching practices, such as the studies of Owens (2014) and Wang (2012), which 
made it clear that teachers’ perceptions appeared clearly in their teaching practices when they were observed in 
the classroom. The second variable is years of experience, such as the study of Al-Anzi and Al-Jabr (2017), Park et 
al. (2016), Ambosaidi et al. (2015), and Smith et al. (2015), whose results did not show statistically significant differ- 
ences in teachers’ perceptions of STEM that attributed to their years of experience. However, the study of Park et 
al. (2016) showed statistically significant differences in teachers with more than 15 years of experience. The third 
variable is gender. The study of Park et al. (2016), Ambosaidi et al. (2015), and Smith et al. (2015) showed statisti- 
cally significant differences in teachers’ perceptions of STEM for male teachers. The fourth variable is educational 
stages. The studies of Al-Anzi and Al-Jabr (2017) and Park et al. (2016) showed statistically significant differences 
in the teachers’ perceptions of STEM which were found in favor of elementary school teachers. 

Changing perceptions is difficult and may take a long time as it has gone through several stages: “awareness, 
development of interest, mental experimentation, actual experimentation, then adoption or rejection” (Al-Saleh, 
2002, p. 10). Thus, this study aimed to know the perceptions of science teachers before applying STEM education 
in schools, or before joining professional development. It may give results that would contribute to guiding profes- 
sional development programs. With a careful extrapolation of the previous studies, it becomes clear that there are 
no local studies that have dealt with the perceptions of science teachers at the secondary level about the concept, 
importance, and mechanisms of STEM education. 


Research Problem 


Scholars in Saudi Arabia have identified challenges to incorporating technology and engineering into science 
textbooks. Efforts have focused on the theoretical aspect of technology and its role in science research more than 
on engineering design or on connecting technology to science problems in society (Al-Ahmad & Al-Bugqami, 2017; 
Al-Beez, 2017; Al-Hamidani, 2017; Al-Ahmadi, 2016); moreover, the natural sciences are rarely integrated (Al-Beez, 
2017; Al-Hamidani, 2017). These shortcomings could negatively affect learners’ understanding of science concepts 
or their abilities to face scientific problems and solve them. Learners in the Trends of the International Mathematics 
and Science Studies (TIMSS, 2019) performed below the average (Al-Shamrani et al., 2016). 

One of the supporting steps of the transformation to a more productive society that can confront problems 
related to science is keeping up with new trends in science education such as STEM education. The interest in this 
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trend was reflected in newly established science centers and STEM education conferences, but despite the interest 
in applying STEM education, there are some challenges. In particular, STEM education tends not to be included 
in teacher preparation courses, and many districts in Saudi Arabia have large numbers of schools and teachers 
(Al-Dossary, 2015). Al-Daoud (2017) recommended identifying science teachers’ perceptions about this trend, and 
Al-Anzi and Al-Jabr (2017) conducted a study to do so; specifically, they studied science teachers’ perceptions of 
STEM education teaching requirements. However, no researchers in Saudi Arabia have examined the meanings 
and importance of STEM for science teachers or their perceptions of integrating mechanisms. This study aimed 
to fill that research gap with a focus on whether science teachers’ perceptions varied according to their years of 
experience, field of specialization, or academic degree. 


Research Questions 


1. What are upper-secondary school science teachers’ perceptions of STEM education in terms of its 
concept, integrating mechanisms, and importance? 

2. Are there any statistically significant differences (a < .05) among upper-secondary school science teach- 
ers’ perceptions of STEM education by specialization? 

3. Arethere any statistically significant differences (a < .05) among upper-secondary school science teach- 
ers’ perceptions of STEM education by years of experience? 

4. Arethere any statistically significant differences (a < .05) among upper-secondary school science teach- 
ers’ perceptions of STEM education by academic degree? 


Research Significance 


1. This study responds to recent trends in science education. 

2. The value and importance of science teachers’ voices in improving science education by identifying 
science teachers’ perceptions of STEM education as previous studies assured (Owens, 2014; Shahzad 
et al., 2017; Wang, 2012). 

3. The results could help in designing professional development programs that meet the needs of sci- 
ence teachers. 


Research Terms 


Perception is a mental process that enables individuals to construct ideas, opinions, or concepts based on their 
personal experiences, feelings, and needs, and teachers’ perceptions have a recognized effect on their classroom 
actions (Choy & Cheah, 2009). Practically, perception in this study refers to science teachers’ opinions about the 
importance of STEM education and the mechanisms of integrating different STEM concepts as well as the influence 
of their perceptions on their teaching performance. 

STEM education refers to curricula that attempt to integrate science, technology, engineering, and mathemat- 
ics into one category based on the connections in the natural world (Stohlmann et al., 2012). For this study, STEM 
education refers to instruction methods that enhance learners’ holistic understanding and utilization of STEM 
concepts and practices. 

Integrating mechanisms refer to methods and forms of integrating science, technology, engineering, and 
mathematics (Bybee, 2013). Mechanisms of integrating the different topics can vary according to the specificity 
of academic subjects or the diversity of educational strategies (NAE & NRC, 2014) and depend on the learning 
context, how science content is organized, and who is teaching. 


Research Methodology 
General Background 
For this descriptive study, the researchers used a questionnaire, a tool with which “all members of the research 


community, or a large sample of them, are questioned to describe the studied phenomenon, in terms of its nature, 
and the degree of its existence, without going beyond that to study the relationship or deduce the causes” (Al- 
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Assaf, 2016, p. 211). A survey was the most appropriate approach for collecting, describing, and interpreting the 
data on science teachers’ perceptions of STEM education. The study was applied during the academic year 2018. 
And it was limited to girls’ public and private upper-secondary schools in the city of Riyadh due to the separation 
between girls and boys in Saudi schools. 


Population and Participants 


According to the statistics of the Riyadh Education Department (2017), the population included 1,754 upper- 
secondary school science teachers from private and public schools in Riyadh, Saudi Arabia. A sample of 700 upper- 
secondary school science teachers was chosen from the population through stratified sampling based on science 
specialization. To ensure that the study takes into account the ethics of scientific research, a brief description of 
the study objectives, procedures, and tools has been sent to the Scientific Research Ethics Committee at King Saud 
University to get their permission. The research objectives have been clarified for the sample in the tool, and it is 
assured that they observe the confidentiality of their data. It was also ensured that the letter sent by the Ministry 
of Education to science teachers did not include any mandatory formula to answer the questionnaire. The informa- 
tion and the questionnaire were sent by the Information Technology Department at the Ministry of Education to 
the sample via text messages. However, although the sample comprised 40% of the population, only 255 teachers 
responded, accounting for approximately 15% of the population; the sample consisted of 92 biology teachers, 78 
physics teachers, and 85 chemistry teachers. In terms of their years of experience, 7% of the teachers had been 
teaching for less than five years, 27% had between five and 10 years of experience, and 66% had taught for more 
than 10 years. By academic degree, 86% of the teachers in this study held a bachelor’s degree, 14% held a master’s 
degree, and only one held a doctorate. All members of the sample were female. 


Instrument and Procedures 


The questionnaire consisted of two parts. The first part asked the respondents about their years of experi- 
ence, specialties (biology, physics, and chemistry), and academic degree; the second part had three sections: 
STEM meaning, importance, and mechanisms for integrating. The number of items in each section was 13 in 
STEM meaning, seven items in mechanism for integrating, and eight items in importance. These items reflected 
the scientific visions of the NEA and NRC (2014) and Bybee’s (2013) integrating mechanisms. The participants 
rated each questionnaire item on 5-point Likert scale (1 = strongly disagree, 2 = disagree, 3 = not sure, 4= agree, 
and 5 = strongly agree). The researchers then divided the responses into one of four categories: high, medium, 
low, and very low. 


Reliability and Validity 


The researchers validated the questionnaire by presenting it to a group of experts in curriculum and instruc- 
tion and calculated the internal consistency reliability using Pearson's correlation coefficient. The results ranged 
between .985 and .609, which indicated that each questionnaire item had a medium to high internal correlation 
and, thus, confirmed the reliability for implementation (Abu Hashem, 2012). Moreover, the researchers administered 
the questionnaire to a pilot sample that comprised 30 upper-secondary school science teachers and calculated a 
Cronbach's alpha of .983, which reflected a high reliability (Allam, 2007). 


Data Analysis 


Various statistical methods were used to analyze the data. Descriptive statistics included frequencies (mean 
and standard deviations) of each item of the questionnaire were calculated to find science teachers’ perceptions of 
STEM education. One-way analysis of variance (ANOVA) was conducted to find the differences in science teachers’ 
perceptions of STEM education according to their years of experience and specialization (biology, physics, chem- 
istry). Scheffe’s test for post-hoc comparisons was used to distinguish the teachers’ differing perceptions according 
to their experience and specialties. A t-test was used to detect differences in science teachers’ perceptions of STEM 
education according to their academic degrees. 
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Research Results 


The findings for the first study question, on upper-secondary school science teachers’ perceptions of STEM 
education, revealed strong agreement in the teachers’ perceptions of the meaning of STEM (M = 4.26). Specifi- 
cally, the two most commonly selected STEM meanings were “connecting learning scientific concepts with issues 
and problems of the natural world” and “preparing a stimulating learning environment to show students’ correct 
and incorrect concepts and discussing them.’ In contrast, the two lowest-ranked STEM meanings are “employing 
simulation software to build predictions about the engineering design performance related to scientific concepts” 
and “developing students’ engineering practices, such as creating or drawing engineering designs, then evaluating 
and developing them to understand or solve scientific problems” (see Table 1). 


Table 1 
Means and Standard Deviations of Science Teachers’ Perceptions of STEM Meaning 


Item M SD Ranking 


STEM education meaning 


3. Connecting learning scientific concepts with issues and problems of the natural world related 4.41 0.651 1 
to environment, energy, health, climate change. 


11. Preparing a stimulating learning environment to show students correct and incorrect 4.40 0.762 2 
concepts and discussing them. 


12. Preparing a motivating learning environment to generate students’ questions. 4.40 0.766 3 


13. Prepare a stimulating learning environment for students to evaluate each other's ideas 4.35 0.784 4 
based on evidence and arguments. 


9. Engaging students in realistic experiences in which they employ science, technology, engi- 4.34 0.674 5 
neering, and math concepts and practices. 


10. Directing the students to determine scientific problems, design and propose solutions, and 4.32 0.725 6 
perform implementation, interpretation and analysis by themselves. 


5. Students’ employing technology to build or apply their scientific knowledge. 4.31 0.666 7 


6. Employing technologies to support dialog and communication between students, research- 4.31 0.739 8 
ers, and scholars. 


2. Students learn science concepts in the light of science phenomena or problems in ways that 4.21 0.794 9 
integrates the related scientific disciplines. 


7. Students use mathematical models to build scientific explanations or engineering designs. 4.16 0.711 10 


1. Lessons are structured in the form of key concepts that connect science, technology, 4.16 0.713 11 
engineering, and mathematics. 


8. Employing simulation software to build predictions about engineering design performance 4.11 0.789 12 
related to science concepts. 


4. Developing students’ engineering practices, such as creating or drawing designs, evaluating 4.07 0.783 13 
them, and developing them to understand or solve science problems. 


1-13 4.26 0.512 


The mean rating for science teachers’ perceptions of how to integrate STEM is 3.84. The mechanism that the 
most participants agreed on was holding activities that support learning mathematics, technology, and engineering 
and help students learn and apply science concepts; the mean rating for this mechanism is 4.24. The integrating 
mechanism that the science teachers agreed on the least was attempting to teach all four STEM concepts in one 
course in collaboration among the different teachers. The mean rating for this integration mechanism is 3.33 (see 
Table 2). 
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Table 2 


Means and Standard Deviations of Science Teachers’ Perception of STEM Integrating Mechanism 


(PP. 546-557) 


Item M SD Ranking 

STEM integrating mechanism 
1. Activities that support learning mathematics, technology, and engineering and help students 4.24 0.693 1 
learn and apply scientific concepts. 
6. Project-based learning that helps students employ concepts and practices in science, 4.13 0.805 2 
technology, engineering, and mathematics. 
7. Create a unit related to contemporary issues or challenges that require students to employ 3.98 0.827 3 
previously studied STEM concepts and practices. 
5. Combining two or three STEM fields to construct or apply integrated scientific knowledge. 3.87 0.839 4 
3. Teaching science, technology, engineering, and mathematics in (separate) coordination 3.73 1.057 5 
between teachers’ specializations to support students’ knowledge and skills to solve specific 
science problems. 
2. Utilizing technology or engineering to build concepts of science and mathematics. 3.63 1.132 6 
4. Teaching science, technology, engineering, and mathematics in one course in cooperation 
between the teachers of these fields to support students’ knowledge and practices to solve 3.33 1.233 7 
specific scientific problems. 
1-7 3.84 0.608 


The results also showed a strong agreement in the science teachers’ perceptions of the importance of STEM, 


with a mean rating of 4.15. The teachers agreed that the most important aspect of STEM is “developing students’ 


skills in solving science problems using creative methods.’ The two importance measures that the science teachers 
rated as the least important are “improving students’ academic achievement” and “developing students’ desire to 
engage in understanding and solving natural world problems related to science, technology, engineering, and 


mathematics” (Table 3). 


Table 3 


Means and Standard Deviations of Science Teachers’ Perceptions of STEM Importance 


Item M SD Ranking 
STEM importance 

6. Developing students’ skills in solving science problems using creative methods 4.38 0. 700 1 
1. Developing students’ knowledge in science, technology, engineering, and mathematics. 4.37 0.625 2 
4. Developing students’ skills in critical thinking. 4.37 0.632 3 
5. Developing students’ skills in decision-making. 4.35 0.681 4 
8. Enhancing students’ awareness of the roles of science, technology, engineering, and 4.33 0.676 5 
mathematics together in addressing major challenges in society. 
2. Developing students’ science, technology, engineering, and math practices. 4.30 0.698 6 
7. Improving students’ academic achievement. 4.30 0.714 7 
3. Developing students’ desire to engage in understanding and solving natural world problems 
related to science, technology, engineering, and mathematics. mee nee : 
1-8 4.34 0.560 
STEM meaning, integrating mechanisms, and importance 4.15 0.488 
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For study question 2, regarding statistically significant differences in the upper-secondary school science 
teachers’ perceptions of STEM education by science, Table 4 presents the univariate test findings. In particular, 
F for STEM meaning is 3.828, and p = .023. However, the univariate tests indicate that the teachers’ perceptions 
of STEM’s meaning and importance and of mechanisms for integrating the STEM concepts differed according to 
their specialization, with F = 3.430 and p = .034. Scheffe’s test for post-hoc comparisons is used to distinguish the 
teachers’ differing perceptions according to their specialties. Univariate testing reveals significant differences; the 
Scheffe's test results are presented in Table 5, which reflects particular differences between the physics and biology 
teachers’ varying STEM perceptions. 


Table 4 
Univariate Test Results for Science Teachers’ Perceptions of STEM Education by Specialization 


First part Source Sum of Squares df Mean Square F p 
Between groups 1.967 2 0.983 
Meaning Within groups 64.73 252 0.257 3.828 023 
Total 66.70 254 
Between groups 1.619 2 0.809 
Integrating mechanisms Within groups 92.12 252 0.366 2.214 11 
Total 93.74 254 
Between groups 1.307 2 0.653 
Importance Within groups 78.33 252 0.311 2.102 124 
Total 79.64 254 
Between groups 1.605 2 0.803 
Total Within groups 58.97 252 0.234 3.430 .034 
Total 60.58 254 


Table 5 
Scheffe’s Post-Hoc Comparisons 


Variables Specialization Mean Difference p 
Chemistry 0.02683 .938 
The concept Physics 
Biology 0.20050 .039* 
Physics Chemistry 0.03980 857 


Science teachers’ perceptions of STEM meaning, integrating 
mechanisms, and importance 


Biology 0.18803 .043* 


Study question 3 asked whether there were statistically significant differences in upper-secondary school 
science teachers’ perceptions of STEM education by years of experience. The findings in Table 6 show an F of .233, 
which was not statistically significant (p = .792), indicating that the teachers’ perceptions of the meaning and im- 
portance of STEM as well as the mechanisms for integrating did not differ according to their years of experience. 
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Univariate Test Results for Science Teachers’ Perceptions of STEM Education by Years of Experience 


First part Source Sum of Squares df Mean Square F p 
Between groups 0.146 2 0.073 
Meaning Within groups 66.556 252 0.264 276 159 
Total 66.702 254 
Between groups 0.106 2 0.053 
Integrating mechanisms Within groups 93.636 252 0.372 142 867 
Total 93.742 254 
Between groups 0.122 2 0.061 
Importance Within groups 79.52 252 0.316 193 824 
Total 79.64 254 
Between groups 0.112 2 0.056 
Total Within groups 60.46 252 0.240 233 792 
Total 60.58 254 


Study question 4, the last question, was regarding statistically significant differences among upper-secondary 
school science teachers’ perceptions of STEM education by academic degree. Table 7 indicates that t was 1.372, 
which was not statistically significant (p = .171). Thus, science teachers’ STEM perceptions did not differ by their 
academic degrees. 


Table 7 
T-test Results for Science Teachers’ Perceptions of STEM Education by Academic Degree 


Y f ttest 
First part a M SD 
experience 
t df p 
Bachelor 4.26 0.510 
The concept 1.722 253 .086 
Postgraduate 4.41 0.508 
Bachelor 3.83 0.609 
Integrating mechanisms 0.620 253 536 
Postgraduate 3.90 0.601 
Bachelor 4.31 0.562 
The importance 1.344 253 .180 
Postgraduate 4.45 0.537 
Bachelor 4.13 0.490 
Total 1.372 253 ATM 
Postgraduate 4.25 0.471 


Discussion 
Engineering is considered a foundation of STEM education. In this study, science teachers strongly agreed 


on the role of engineering in constructing and applying science concepts but ranked it last in importance 
in terms of the meaning of STEM. This could be attributed to the fact that science curricula are based on the 
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National Scientific Education Standards, which place less emphasis on engineering practices than the NGSS. 
The teachers herein agreed on the role of mathematics in learning and applying science concepts. This result 
is consistent with findings by Al-Anzi and Al-Jabr (2017) and Ambosaidi et al. (2015) in that science teachers 
perceive considerable integration between science and mathematics. 

In addition, the science teachers in this study agreed that “scientific activities that support learning math- 
ematics, technology, and engineering and help students in learning and applying scientific concepts” were 
the most important mechanism for integrating the concepts of STEM in education. It was attributed to the 
aforementioned similarity between this mechanism and the structure of activities in science curricula. Some 
studies have indicated moderate levels in terms of the incorporation of mathematics into teaching from science 
books (Al-Hamidani, 2017; Al-Beez, 2017) but a minimal inclusion of technology and engineering (Al-Ahmad 
and Al-Buqami, 2017; Al-Beez, 2017; Al-Hamidani, 2017; Al-Ahmadi, 2016). The result herein agrees with the 
aforementioned findings and with those of Wang (2012), who found that science teachers recognized technol- 
ogy and math as tools that can help learners better understand science matters. 

Furthermore, it was attributed to the high agreement among the science teachers in this study that “project- 
based learning helps students in employing concepts and practices in science, technology, engineering, and 
mathematics” to the existence of professional development programs for science teachers for project-based 
learning; it is, thus, not a new concept. This finding is consistent with those from Sandall (2016), Owens (2014), 
and Turner (2013) in that project-based learning has a fundamental role in integrating science, technology, 
engineering, and mathematics. 

It was also considered that science teachers’ moderate agreement on the importance of creating “a unit 
related to contemporary issues or challenges, which requires students to employ previous studied concepts 
and practices in science, technology, engineering and mathematics” was related to how science curricula often 
end with an investigative activity that aims to employ two or more STEM concepts and practices. Consistent 
with the findings herein, Al-Hamidani (2017) found statistically significant differences in incorporating STEM 
activities into physics teaching, in particular, for lab activities because these were designed in the form of proj- 
ects. Additionally, it might be easier for teachers to guide students in applying previously taught concepts than 
to attempt to build new knowledge through course projects, a method Slough and Milam (2013) identified as 
common in school projects. 

Moreover, the finding of science teachers’ perceptions about the importance of STEM, for which there 
was a high agreement among them, is in agreement with previous studies. Park et al. (2016) found that teach- 
ers believed STEM teaching enhanced students’ learning, creative thinking, and personality building, and the 
teachers in Turner's (2013) investigation believed that STEM teaching developed 21st-century skills. Wang (2012) 
also indicated that science teachers believed that STEM teaching gave learners the opportunity to apply their 
knowledge in life situations. 

In addition, it was attributed to the fundamental differences between physics and biology teachers’ per- 
ceptions of the meaning and importance of STEM and of integrating mechanisms to the fact that the nature 
of physics ties it more closely to concepts of geometry, algebra, and arithmetic than to biology. Abdel-Hamid 
et al. (2015) indicated that 46 engineering skills related to the following content should be incorporated into 
physics textbooks: vectors and analysis of forces and field, followed by algebraic skills (24 skills), arithmetic (11 
skills), and statistical (5 skills). 

In contrast, the absence of significant differences between science teachers’ perceptions of STEM educa- 
tion by their years of experience or academic degrees can be attributed to the fact that the concept of STEM is 
based on philosophical and logical foundations that stem from the unity of scientific knowledge, which cannot 
be partially constructed or applied. Furthermore, “STEM” is a modern term, but its roots belong to the theory of 
constructivism, which existing science curricula are based on, and this makes it acceptable to science teachers 
irrespective of their education or years of experience. This finding is in agreement with Ambosaidi et al. (2015) 
and Smith et al. (2015) in that science and agriculture teachers did not differ in their perceptions of the concept 
of STEM according to their years of experience. However, the results of this study contradict Park et al’s (2016) 
finding that teachers who had 15 or more years of experience were more accepting of the concept of STEM and 
its importance than teachers who had fewer than 5 years of experience. This difference may be attributable to 
the types of schools that were studied; specifically, Park et al. (2016) studied schools with a STEM focus. 
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Conclusions and Implications 


An interest in STEM education, as it supports the building and application of scientific knowledge to solve 
the problems of society and develop its economy, was an impetus to reveal the perceptions of science teachers of 
STEM. This study aimed to recognize science teachers’ acceptance of STEM requirements and to guide professional 
development programs. However, due to the low response of the upper-secondary school science teachers to 
the research tool that came at a rate of 37.5% of the sample, the generalization of this study results became limit. 
Moreover, the sample of this study were female which limits the ability to generalize the results. 

The results revealed a high level of agreement regarding the concept and importance of STEM. However, there 
were some difficulties related to the teachers’ preparation. Thus, this study recommends intensifying professional 
development programs in terms of employing engineering. This will lead to learning science that goes beyond 
a general engineering framework and concepts. The programs should relate to the science content teachers are 
already teaching. 

This study showed that in a rather high standard deviation values in some science teachers’ perceptions of 
the integration mechanisms. Thus, an analytical study can be recommended to reveal mechanisms of integration 
in studies that have researched the effectiveness of STEM education on learning outcomes. 

For future studies, researchers should explore the relationship between science teachers’ perceptions of the 
concept of STEM and their teaching practices. Further, analytical studies should be conducted to identify mecha- 
nisms for integrating STEM concepts into science teaching using dependent variables such as learners’ interest in 
STEM fields and learning. 
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Abstract. This study examined the impact 
of the Augmented reality (AR) use on Jorda- 
nian 6th grade students’ attitudes towards 
science and their learning outcomes. A 
quantitative quasi-experimental study is 
preceded with the Pre-test and Post-test 
control group design model, where 24 
students who identified to have special 
needs participated in this study and were 
randomly divided into two groups. Two 
groups, control group 12 students were 
taught conventionally, and 12 students 
were designated as the experimental group, 
they used the AR technology for four weeks. 
The two scales used in this research were 
reliable and validated. The results show 
significant results for the AR technology 

in enhancing student learning outcomes. 
Additionally, results supported that AR 
technology has the potential to enhance 
students with learning disabilities positive 
attitudes. The result shows that AR technol- 
ogy helped students in promoting positive 
attitudes towards students and enhance 
students learning outcomes. 
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COMPARISON OF AUGMENTED 
REALITY AND CONVENTIONAL 
TEACHING ON SPECIAL 
NEEDS STUDENTS’ ATTITUDES 
TOWARDS SCIENCE AND THEIR 
LEARNING OUTCOMES 


Turki Alqarni 


Introduction 


The study of Science is a process that involves several complexities 
relating to problem identification, problem investigation, hypotheses for- 
mulation, data collection method planning, hypotheses testing, data collec- 
tion, obtaining results and making conclusions (Meerah, 1998; Saidin et al., 
2015). Literature highlighted that there is a decrease in science achievement 
among students (Bicer & Lee, 2019). In other related studies, abstract and 
ambiguous course contents lead to misunderstanding and low achievement 
levels among students (Erbas & Demirer, 2019). In the modern educational 
system, 3.5% of the students were evidenced to display reading problems, 
which constitute one of the top common and current learning disabilities 
displayed among them (Karamanoli & Tsinakos, 2015). Additionally, among 
disabled students, creative interactive activities, visual presentations, project- 
based learning, and school experiments that are centered on engagement 
and activities of students are of significant benefit (Obradovic et al., 2015). 
In this regard, there is a need to stimulate learning disabilities through 
the use of visual and perceptive activities (Rega & Mennitto, 2017), with 
literature evidencing that school students are slowly losing their interest 
and motivation towards learning science subjects (e.g., Cimer, 2012; Potvin 
& Hasni, 2014). In the face of the tireless efforts made by the educational 
systems around the globe in supporting special needs students, challenges 
still abound when it comes to their learning, which are related to the teach- 
ing approach, services provided, teachers with lack of experience with the 
students and identification of students suffering from learning disabilities 
(Alnahdi et al., 2019; Binmahfooz, 2019). As students with learning disabilities 
move from one stage to the next, academic requirements and expectations 
also increase. This is particularly true because evidence shows that learn- 
ing disabilities have divergent characteristics (Billingsley et al., 2018), while 
more has to be researched on instructional practices enhancing learning 
outcomes, achievement and level of engagement of students with learning 
disabilities (Billingsley et al., 2018). 

Therefore, promoting positive attitudes towards science courses among 
students is a crucial goal among academicians (Musalamani et al., 2021; 
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Toma et al., 2019). Among educators, there is an increasing trend to develop and implement evidence-based 
practice in the hopes of preparation individuals with and without learning disability to live normal, productive, 
and enriching lives, through refining methodologies refinement and creation (Kellems et al., 2020). Regardless of 
the countless approaches to teaching, which have been adopted to enhance learning among students with and 
without disabilities in science courses, authors on the topic indicate inconclusive findings as to the effectiveness 
of intervention (Kellems et al., 2020; Savelsbergh et al., 2016). The education innovation literature brings forward 
advance technology as an innovation that could enhance learning outcomes and successful achievements of 
students in both groups (with and without learning disabilities) (Kellems et al., 2019; Savelsbergh et al., 2016). 
Advancements in technology have extended the limitations of teaching and learning activities and the develop- 
ment of methods to deliver new courses (e.g., e-learning, virtual lectures, and augmented reality (AR) (Kellems 
et al., 2019; Khan et al., 2019; Savelsbergh et al., 2016; Yot-Dominguez & Marcelo, 2017). Specifically, augmented 
reality (AR) has been deemed as a promising innovation that paves the way for teaching and learning opportu- 
nities, enhances students’ success, those with and without disabilities (Kellems et al., 2019), and it is presently 
transforming the interaction and engagement of people with inanimate objects (Weng et al., 2020). On the basis 
of research studies dedicated to the AR technology, AR can promote teaching and learning outcomes in light of 
achievement, attitude, confidence, motivation, interest, spatial ability, engagement and ultimately, satisfaction 
of students (e.g., Akcayir & Akcayir, 2017; Weng et al., 2020). The AR technology application has been generally 
found to enhance experiences in distinct learning environments offering learning activities (Yuen et al., 2011). 
Augmented reality (AR) covers an extensive range of technology projecting computer generated contents (text, 
images, and videos) onto real world perceptions (Yuen et al., 2011). In AR technology, a combination of the physical 
and virtual world is directed towards improving the environment of the user, supported by additional information 
in the form of camera-generated images, videos and audio through the use of computer and mobile technolo- 
gies (Sommerauer & Muller, 2014). Yuen et al. (2011) presented a detailed explanation and discussion concerning 
AR technology and the related applications. In other studies, Akcayir et al. (2016) contended that AR technology 
application in the field of education has become even more feasible in the current times with new applications 
facilitated through computers and mobile devices making them affordable, compared to their predecessors that 
needed sophisticated equipment (e.g., head-mounted displays). 

Through the combination of real and virtual objects, AR enables the visualization of abstract concepts and 
complex spatial relationships and the experiencing of phenomena in a way that cannot be experienced in the 
real world, and thus, reinforcing the interested towards education (Wu et al., 2013). There are several forms of 
AR applications like AR books, AR gaming, discovery-based learning, object modeling, skills training (Yuen et al., 
2011). In AR books, the digital content is combined with physical contents through the physical book’s augmenta- 
tion with 3D objects, voice and elements of multimedia, enabling the mitigation of gap between the virtual and 
physical realm (Yuen et al., 2011). Furthermore, AR books facilitate immersive learning, in-depth understanding 
and encourages the motivation, participation, and engagement of learners (Chen, 2006; Shelton & Hedley, 2002). 
In the teaching of science such as biology, effective use of visualization methods like 3D materials, actual real-life 
objects, videos and other technology assist the learners to learn the abstract and invisible concepts of the subject 
(Cimer, 2012). AR can also be combined, with assistive or instruction technology, according to which the principles 
of learning design enable students with learning abilities to effectively and successfully learn (Walker et al., 2017). 

In this regard, several works have explored AR technology and reported that it affects learning environment 
(Akcayir et al., 2016; Kellems et al., 2020). In Mc Mahon et al’s (2016) study, the authors worked with postsecondary 
students suffering from intellectual disabilities, with the use of AR to instruct them on the vocabulary of science, 
whereas Chang et al. (2013) used AR with students with disabilities to teach them work-related skills. Chen Lee, 
and Lin (2015) also used the same to illustrate ways of identifying emotions, and Kellems et al. (2020) involved 
middle school students in their use of AR to teach mathematics. Despite the adopted teaching approaches di- 
rected towards enhancing learning achievements among students (with and without learning disabilities), works 
dedicated to examining students’ outcomes through the use of AR technology are still few and far between, with 
reported inconclusive findings with regards to the effectiveness of intervention (Kellems et al., 2020; Savelsbergh 
et al., 2016). Prior studies have only begun to touch the potential of AR as an instructional technique, specifically 
with those having intellectual disabilities and as such, studies are required to examine its effectiveness when 
used with SLD students (Kellems et al., 2020). Furthermore, the limited studies of AR studies in light of student 
attitudes should be noted. According to prior studies (Kellems et al., 2020; Rega & Mennitto, 2017; Walker et al., 
2017), research studies dedicated to AR as an evidenced-based approach for instructing students with special 
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needs are still few and far between, and as such, the present study primarily aims to examine the viability and 
feasibility of using AR in textbook design for basic science in an attempt to enhance learning outcomes and to 
determine attitude towards basic science among students with learning disability. Hence, the present study 
contributes to literature on knowledge-based interventions, providing positive learning outcomes, as it examines 
the attitudes of students with learning disabilities towards basic science course, using AR approach, which is the 
first of this endeavor. This is significant as attitudes play a key role in successful integration of AR technologies in 
educational settings. This study sought to answer the following research question” how AR application promotes 
positive attitudes towards science and enhances the learning outcomes among special need students.’ This study 
hypothesized no significant differences in students’ attitudes towards science and their learning outcomes be- 
tween AR and the control group. 


Augmented Reality (AR) in the Learning Environment 


AR studies point towards its high significance in the 21* century for learners with disabilities (Kellems et 
al., 2020), with related technology enabling teachers to succeed in satisfying the requirements of students with 
disabilities by allowing them to experience the real world combined with the virtual one, all the while linking 
it with reality (Karamanoli & Tsinakos, 2017). Nevertheless, research focused on the use of AR interventions for 
students with disabilities has yet to be extensively conducted as evidenced by the recommendations of Gupta 
et al. (2019) and Kellems et al. (2020). Among the studies that examined intellectually challenged elementary 
students using AR for teaching fundamental matching skills successfully was carried out by Richard et al. (2007), 
while Smith et al. (2016) used AR to examine teaching navigation skills to students with the same disabilities. 

In Mc Mahon et al-s (2016) study, the authors worked with postsecondary students suffering from intel- 
lectual disabilities, with the use of AR to instruct them on the vocabulary of science, whereas Chang et al. (2013) 
used AR with students with disabilities to teach them work-related skills. Chen Lee, and Lin (2015) also used the 
same to illustrate ways of identifying emotions, and Kellems et al. (2020) involved middle school students in their 
use of AR to teach mathematics. Prior studies have only begun to touch the potential of AR as an instructional 
technique, specifically with those having intellectual disabilities and as such, studies are required to examine 
its effectiveness when used with SLD students (Kellems et al., 2020). 

Based on prior studies, AR technology is capable of enhancing educational outcomes and achievements 
(e.g., Chiu et al., 2015). To begin with, Dede (2009) revealed the relevance of AR in students’ engagement in 
explorations of the actual physical world and Klopfer and Squire (2008) evidenced it enabling the experience 
of scientific experiments like chemical reactions that cannot be easily experienced in the actual world. In the 
same way, AR was also evidenced to enable the visualization of concepts like airflow/magnetic fields, and events 
through the display of virtual elements over actual objects (Dunleavy et al., 2009; Wu et al., 2013). 

Added to the above, Singh et al. (2015) showed that AR enhances the students’ knowledge and skills and 
revives their educational interest, and this according to El-Sayed et al. (2011) is true more so compared to other 
technologies. AR boosts the motivation of the students, their gaining of optimum skills of investigation and 
steers clear of providing them conceptual fallacies experience (Sotiriou & Bogner, 2008). AR is unique in its 
visualization features (Erbas & Demirer, 2019) and it has been, time and again, proven to be an effective tech- 
nology to visualize and provide concrete instances of abstract concepts (Erbas & Demirer, 2019). In the same 
way, Ab-Aziz et al. (2012) focused on the effectiveness of two technological trends on the learning of students 
in the special education classes of Malaysia and found augmented reality to benefit their process of learning. 
According to them, AR does provide advantages to the learning process. Moreover, AR, as a hand phone app was 
revealed by McMahon et al. (2016) to assist in learning science vocabulary and enhanced general vocabulary 
among students with disabilities. This was supported in Lin et al’s (2016) study, who revealed that the app use 
among students with ADHA and reading disabilities enhanced their word recognition skills. Lastly, Almutairi 
and Al-Megren (2017) developed an app with AR for the teaching of Arabic to deaf primary school children. 
They combined video, images and audio with AR and found that teachers and parents of deaf children made 
effective use of many resources. 

Attitudes refer to the response of individuals towards objects and conditions, generating, driving and im- 
pacting situations (Inceoglu, 1985). Attitudes refer to tendencies rather than behaviors that direct individuals 
to behave in specific ways (Sirakaya & Cakmak, 2018). In the context of technology, positive attitudes towards 
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it directly affect its use. The attitudes of individuals towards new technology adoption differ and because of 
such difference, the process of integration may culminate in adoption/rejection of technologies (Akca & Ozer, 
2012). According to Sirakaya and Cakmak (2018), the attitudes of students towards new technology will affect 
its use (effectiveness and productivity) in the classroom and hence, it becomes a must to identify the attitudes 
of students towards AR applications to ensure its successful use in learning. In effect, attitude towards AR is 
significant in making sure that it is acquired and disseminated in schools. AR is generally utilized in the educa- 
tion setting to positively motivate students and teachers (Sumadio & Awang, 2010) and to increase the learn- 
ing experience excitement (Ab-Aziz et al., 2012). It assists abnormal kids to obtain and develop cognitive and 
motor skills, making education excited, fun, interactive and compelling (Dhamdhere et al., 2019). Finally, prior 
studies (Kellems et al., 2020; Rega & Mennitto, 2017; Walker et al., 2017), dedicated to AR as an evidenced-based 
approach for instructing students suffering from learning disabilities are still few and far between, and as such, 
the present study primarily aims to examine the viability and feasibility of using AR in textbook design for basic 
science in an attempt to enhance learning outcomes and to determine attitude towards basic science among 
students with learning disabilities. This main aim is achieved by determining the following research questions; 

1. What is the attitude of students with learning disabilities towards AR application? 

2. What is the effect of using AR in basic science course on the learning outcomes of disabled students? 

3. Dothestudents who learn using AR learning approach have significantly better attitude towards learn- 

ing basic science compared to those using traditional learning approach? 
4. Dostudents in the experimental group score higher attitude scores in the pre-test compared to post-test? 
5. Do students in the experimental group score higher learning outcomes in the pre-test compared to 
post-test? 


Research Methodology 
General Background 


This research employed a quantitative quasi-experimental approach with equivalent control group pretest 
and posttest design (Creswell, 2014), to explore the AR effectiveness in promoting positive attitudes towards sci- 
ence and enhancing the learning outcomes among 6" grade special needs students. The students were selected 
from the schools following Jordanian Ministry of Education (MoE). The survey was used as it has been generally 
used to determine characteristics like views, abilities, beliefs, attitudes, expectations, and thoughts (Jdaitawi, 2020; 
Creswell, 2012; Fraenkel & Waleen, 2006; Sirakaya & Cakmak, 2018; Jdaitawi, 2020). According to Buyukozturk et 
al. (2008), survey studies primarily aim to present the case under study. The study group consisted of 24 grade 
6th students in primary schools that have special needs to examine AR supported instructional experience. The 
students in the group were identified using purposive sampling method, where the sample is determined on the 
basis of the research purpose (Fraenkel & Wallen, 2006). This research used the criterion of experience in AR sup- 
ported instruction to determine the study group. The instruction was provided in a span of 4 weeks divided into 
4 units according to the 6" grade science curriculum of the academic year 2018/2019, using AR application. The 
students were categorized into two groups (AR groups and Control group). 


Research Setting and Sampling 


In this research, schools having students with special needs were selected in the middle government schools 
in the Middle governorates of Jordan. It made up of a mixture of different needs. 6" grade students were selected 
from several schools and were randomly assigned as the research sample. Afterword, two groups with 24 students 
were selected. 12 students were selected as the experimental group taught using the AR application to learn sci- 
ence, and the other 12 students taught using the conventional approach without AR. Thus, a total of 24 6" graders, 
aged 12-13, took part in this research as they were accessible and available to the researcher, as well as only 24 
students were identified by the schools as special needs students. The sample is tabulated in Table 1 below. The 
study followed the guidelines and ethical principles stipulated by Jordanian Ministry of Higher Education & Scientific 
Research. Necessary permission was obtained from the schools where the research was conducted. Furthermore, 
the researcher informed the participants that the data will be used for the research purpose only. 
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Table 1 


Descriptive Statistic of the Participants 


Characteristic N % 
AR group 12 50 

Group 
Control 12 50 


Research Instruments 


This research used learning outcomes test and Augmented Reality Applications Attitude Scale on primary 
school students to collect data. With regards to Augmented Reality Application Attitude Scale, it was adopted from 
the study by Kucuk et al. (2014) to determine the students’ attitudes towards AR application in their learning process, 
with 15 items categorized into three factors, each containing items. Prior studies used the scale and confirmed its 
reliability and validity (Kucuk et al., 2014). The scale measuring attitude contains three factors, namely satisfaction 
from use, anxiety to use and willingness to use, and they were measured using 15 items along a 5-point Likert 
scale ranging from 1 (strongly disagree) to 5 (strongly agree). Moving on to learning outcomes scale, the scale 
was adopted and developed based on several studies, namely Astuti et al. (2019) study, Di-Serio et al. (2013), and 
Mundy, Hernandez and Green's (2019) study. This scale was used to identify the knowledge levels of students with 
learning disabilities and their cognitive and effective outcomes relating to the process of learning. There were 11 
items measured on a 5-point Likert scale, ranging from 1 (strongly disagree) to 5 (strongly agree). 


Validity and Reliability 


The original version of research instruments was developed in English, since students who participated in this 
research are native language being Arabic, the instruments were translated and validated translators for Arabic 
speaking students. However, the instruments were translated by two bilingual speakers who are specialists and 
a PhD holder. The translated version was given to 5 educational experts for instrument validations, most of them 
working at the university and some with special needs students. The experts highlighted some issues, and their 
feedbacks were accepted and incorporated and were corrected accordingly. The scale’s internal consistency reli- 
ability was found to be .81, indicating that it is a valid and reliable scale used to assess the attitudes among primary 
school students with learning disabilities towards the use of AR applications. The calculated internal consistency 
reliability coefficient of the scale was revealed to be .73, indicating its validity and reliability in assessing primary 
school students with learning disabilities towards AR applications usage. 


AR Learning Material 


Students’attitudes towards AR applications in educational environments were determined by providing them 
with the experience in a span of 4 weeks through 8 lessons in 6" grade, with the aim of providing students and 
teachers with 3D displays of science lessons using school lab. AR application was developed by taking the acquisi- 
tions an activity of the “Space” unit included in the 6" grade science class, based on the activities in the textbook. 
Initially, in this study, the research obtained the opinions and feedback of 2 field experts, 2 teachers and 3 technical 
experts during the process of the AR application development. The study conducted pre-test and post-test evalu- 
ation. There were 24 students with disabilities at the schools. In a ten-minute pre-test organized in the first day, 
students were given a question to answer without access to any information material or reference classes. Next, 
they were given the questionnaire related to the study variables. Then, the students were split into two groups. The 
first group was a control group and the teacher taught them using traditional method, which began by explaining 
the main ideas and supportive activities and ending by assigning assignment and discussion. The second group 
was exposed to AR classes in their learning activity. The AR classes were introduced into several lessons in science 
curriculum. The introduction lesson involved assistant from the teacher as a moderator to demonstrate the activity 
to students and detailing their difficulties in the subject’s context in the form of a visual. Then the students would 
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have to repeat the activity without assistance. Then the teacher introduced AR activity to make the topic easier. 
For each task, participants were accompanied by after the learning activities, the post-test questionnaires were 
distributed to students for completion. Experimental groups were taught using AR application. Figure 1 showed 
the activities performed in the AR application. 


Figure 1 
The AR Activity used in the Study 


Data Collection 


This research used learning outcomes test and augmented reality applications attitude Scale on primary school 
students to collect data. With regards to augmented reality application attitude scale, it was adopted from the 
study by Kucuk et al. (2014) to determine the students’ attitudes towards AR application in their learning process, 
with 15 items categorized into three factors, each containing items. Prior studies used the scale and confirmed its 
reliability and validity (Kucuk et al., 2014). The scale measuring attitude contains three factors, namely satisfaction 
from use, anxiety to use and willingness to use, and they were measured using 15 items along a 5-point Likert scale 
ranging from 1 (strongly disagree) to 5 (strongly agree). The scale’s internal consistency reliability was found to 
be 0.81, indicating that it is a valid and reliable scale used to assess the attitudes among primary school students 
with learning disabilities towards the use of AR applications. 

Moving on to learning outcomes scale, the scale was adopted and developed based on several studies, namely 
Astuti et al. (2019) study, Di-Serio et al. (2013) study, and Mundy, Hernandez and Green's (2019) study. This scale 
was used to identify the knowledge levels of students with learning disabilities and their cognitive and effective 
outcomes relating to the process of learning. There were 11 items measured on a 5-point Likert scale, ranging from 
1 (strongly disagree) to 5 (strongly agree). The calculated internal consistency reliability coefficient of the scale 
was revealed to be .73, indicating its validity and reliability in assessing primary school students with learning dis- 
abilities towards AR applications usage. 


Data Analysis 


The data was examined for checking the normality (Skewness and Kurtosis and outlier Mahalanobis) cases 
using several indicators such as (Judd et al., 2017; Tabachnick & Fidell, 2007). However, the results proved to be 
normal and outlier cases were identified. Descriptive statistics such as mean Mand standard deviation SD and other 
statistical tests such as independent sample t-test, ANOVA and ANCOVA were involved in this study to identify the 
possible mean differences between the AR group and control group. 


Research Results 
Prior to testing the hypotheses (there is no significant difference in the learning outcomes posttest in sci- 


ence course between AR group and control group), the independent sample T-tests were conducted on the 
independent samples to identify the statistical equivalence of the groups. The dependent variables included are 
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learning outcomes and attitudes towards science. In the initial set of statistical tests, the difference between the 
experimental and control groups in pre-test learning was obtained based on the level of significance (.05). From 
the result, insignificant differences in t-test were found between the groups based on pre-test of learning outcome 
scores (F = .003, p = .955, t= .543, p = .599), but F-test for the equal variance was significant. The mean score of 
the total pre-test sample of learning outcomes is (M = 2.60), with standard deviation of (SD = 0.307). In order to 
control the significant differences between the groups in the pretest result, ANCOVA analysis was conducted. The 
analysis was run for the variable learning outcomes of the groups and for the first set of analysis, significant main 
effects were found for student groups with the dependent variable; for the experimental group the mean square 
was (value = 3.514, F=17.840, p = .0001 <.05) as shown in table 3. The experimental group showed higher scores 
for total learning outcomes with a (M = 3.63, SD = 0.481), compared to its controlled counterpart that had a mean 
of (M = 2.87, SD = 0.380) as shown on table 4. 


Table 2 
Summary Statistics for Learning Outcomes Variable Pre-test Scores 


Variable Experimental Control Total 


M 2.63 2.56 2.60 
SD .300 324 307 


Learning Outcomes 


Table 3 
Results of ANCOVA for Between-Subjects Effect of the Learning Outcomes Post-test 


Source Ms of F p 
Corrected Model 1.758 2 8.928 .002 
Intercept 3.124 1 15.863 .001 
Group 3.514 1 17.840 001 
Learning Outcomes Pretest .004 1 020 889 
Error 4.136 21 
Total 261.744 24 
Corrected Model 7.653 23 

p<.05 
Table 4 


Summary Statistics for Learning Outcomes Variable Post-test Scores 


Variable Experimental Control Total 


M 3.63 2.87 3.25 
SD 481 380 576 


Learning Outcomes 


To determine the statistical equivalence of the groups in the second hypothesis (there is no significant dif- 
ference in the students’ attitudes post-test towards science course between AR group and control group), the 
independent samples were exposed to independent sample t-test prior to hypotheses testing. In the first statistical 
test, the difference between experimental and control groups based on their pre-test attitudes towards science 
was identified based on .01 level of significance. The t-test results supported insignificant differences between the 
groups (F = .906, p .351, t= 1.227, p=.233). The post-test scores of the two groups in terms of attitudes towards 
science indicated significant differences, with the following result for the experimental group (F = 3.159, p =.089, t 
= 2.372, p = .027 <.05) as shown in table 5. The mean score of attitudes in the control group (M = 2.78, SD =0.527) 
is relative to the experimental group (M = 3.20, SD = 0.304) as presented in table 6. 
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Table 5 


Results of Independent Sample t-test for Variables Attitudes towards Science Post-test 


Variable F-value Sig. value t of p 


Attitudes towards Science 3.159 089 2.372 22 027 


Table 6 
Summary Statistics for Attitudes towards Science Variable Post-test Scores 


Variable Experimental Control 


M 3.20 2.78 
SD 0.304 0.527 


Attitudes towards Science 


Based on the results of the ANOVA test, significant differences existed between the groups in light of their 
attitudes towards science. ANOVA results showed the experimental group to obtain the following values (MS = 
1.042, F = 5.626, p = .027 < .05). 


Table 7 
Results of ANOVA for the Effect of the Attitudes towards Science Post-test 


Source Sum of Squares Mean df Square F p 
Between Group 1.042 1 1.042 5.626 027 
Within Group 4.074 22 185 
Total 5.115 23 
p<.05 


Moving on with testing of the test difference hypothesis (i.e., there is no significant difference in the level of 
pre-test and post-test attitude scores among experimental groups), the study carried out paired sample t-test to 
determine whether there is a significant mean score of the students with special needs attitudes in the pre-test and 
post-test scores. The overall attitude in pre-test mean score was found to be (M = 2.45), with standard deviation 
of (SD = 0.459), while the overall attitude in post-test mean score was (M = 2.99), with standard deviation of (SD = 
0.471) as shown in table 8. The mean score increased based on the pre-test and post-test mean scores. 


Table 8 
Summary Statistics for the Experimental Group Attitudes Towards Science Scores 


Variable Post-test Pre-test 


M 2.99 2.45 


Attitudes towards Science 
SD 0.471 0.459 


The research conducted a comparison of the overall mean square using the paired sample t-test and it was 
found to be significant at (t = 4.666, df = 23, p = .01), with a difference of (.544), indicating a significant increase 
(p =.05) in the experimental group's mean in light of the attitude of the students towards science as shown in table 9. 
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Table 9 


Results of Paired Sample t-test for Variable 


Variable t df p 


Attitude towards Science 


Experimental group 4.66 23 001 


Moving on with testing of the (i.e., there is no significant difference in the level of pre-test and post-test earn- 
ing outcomes scores among experimental groups), paired sample t-test was carried out to know if the mean score 
in the learning outcomes of students was significant based on pre-test and post-test scores. The overall learning 
outcomes of the experimental group's pre-test had a mean of (M = 2.60), with standard deviation of (SD = 0.307), 
while in the post-test the mean was (M = 3.25, with standard deviation score of (SD = 0.576) as displayed in table 
10, indicating increased scores following the implementation of learning technique. 


Table 10 
Summary Statistics for the Experimental Group Learning Outcomes Scores 


Variable Post-test Pre-test 


M 3.25 2.60 
SD 0.576 0.307 


Learning Outcomes 


The research compared the overall mean score of the pretest and posttest for the experimental group using 
paired sample t-test, after which it was found to be significant at t (4.842, df= 23, p =.01), with a mean score differ- 
ence of (.651), and a significant increase of (p = 0.05). The detailed results are presented in table 11. 


Table 11 
Results of Paired Sample t-test for Learning Outcomes Variable 


Variable t df p 


Learning Outcomes 
Experimental group 4.842 23 .001 


Discussion 


The aim of this research was to examine the AR application’s effectiveness in the attitude of primary school 
students suffering from learning disabilities, towards basic science, and to identify the different effects of AR 
technology on their learning outcomes. The AR application development had its basis on the science teaching 
material specified in the research framework and the development was made under the feedback of field experts, 
technical experts, and teachers. Prior to collecting data, students were instructed on basic science lessons in 
the span of two weeks through AR technology, which basically provided them a learning experience through it. 
Based on the results, the students with learning disabilities showed a positive attitude towards the application 
during the instruction. AR essentially facilitated a positive environment in the classroom that is distinct from 
the traditional classroom, which was assumed to promote the positive attitude of students towards learning 
basic science. The research results showed that AR application assisted in the students’ development of positive 
thoughts about science lessons and in enhancing their attitudes towards their learning environment. In other 
words, the use of AR technology benefited the educational environment and provided interactive and proactive 
learning environment through reality enhancement (Sirakaya & Cakmak, 2018). According to literature findings, 
AR teaching material provided positive contributions to the level of attitudes, willingness, enthusiasm, moti- 
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vation, self-confidence, academic achievements and persistence and readiness of students. Lack of evidence 
concerning the direct contributions of AR to students with special education needs did not negate them. This 
highlights the contribution of technology-centered environments in education (Alghabban et al., 2017; Ayres 
et al., 2009; Bakker et al., 2016). 

To begin with, Akir and Korkmaz (2019) showed that AR technology materials enhanced the interest of stu- 
dents with special needs towards the studied subject, and Delello (2014) revealed that AR technology improved 
the student's interest towards lessons. AR application also improved the interest of students and directed it to- 
wards materials of learning (Ibili & Sahin, 2013), which significantly heightened their attention towards learning, 
particularly students with vision and hearing problems, retarded growth and behavioral problems, and those 
with special education needs (Cakir & Korkmaz, 2019). In the same line of study, AR assisted students to learn 
more easily and more effectively, increasing their interest towards learning procedures (Ho et al., 2011). The find- 
ings indicated significant differences between students exposed to AR technique of learning and conventional 
learning, and the total score of attitudes towards basic science subject also had significant differences between 
the two groups, with the AR group being higher. This is indicative of the fact that AR application usage could be 
effective in enhancing the interest and attitudes of students with learning disabilities. This was supported by 
Chen et al’s (2016) finding that showed AR to be capable of assisting students with ASD to display their feelings 
and status and be aware of different circumstances. Also, Pradibta (2018) revealed the way AR enlightened the 
students learning and directed their attention towards the learning process, while Karamanoli and Tsinakos 
(2016) revealed the advantages that AR tool can provide students in terms of transforming their learning process 
to a more stimulated and entertaining process. It paved the way for dyslexic students to be more interactive 
while learning and gaining new experiences. The findings of this research are in line with that of prior studies in 
literature, where students suffering from disabilities can be made more inclined towards learning in a technol- 
ogy enriched classroom surrounding. 


Learning Outcomes 


The findings of this research revealed that students with special needs enhanced their learning outcomes 
through the AR application used for basic science lessons. AR technology usage significantly affected the out- 
comes of students’ learning, indicating that the use of such technology was an effective tool, particularly when 
used with disabled students. Notably, AR supported the learning outcomes of the students in a way that tra- 
ditional classroom was not able to. In the same line of study, Delello (2014) stated that AR had a key role in the 
learning of students, urging their participation and interaction in class and enabling their deeper understand- 
ing of subjects. AR technology was thus an effective support learning and teaching tool in the education realm 
(Hsu, 2017; Tian et al., 2014), through the advantages reaped from it. In addition, AR technology was capable of 
easing learning among students and their understanding of the content material, which in turn, affected their 
learning capabilities. It familiarized students with the study and its activities boosting their interaction in the 
process of learning. In fact, Karamanoli and Tsinakos (2016) indicated that AR enhanced the learning activities 
provided to students, especially those with learning disabilities from simple and static to interactive. The dif- 
ferences in pre-test and post-test may be attributed to the visual support learning mentioned in Cimer’s (2012) 
study. AR technology facilitated visual-aided learning and teaching, where students could easily learn through 
visual aids. It also facilitated the participation of the students, their concept retention through the tools (visual 
and audio), promoting their connection of ideas and concepts. 

According to the research findings, significant differences were found between the two groups in light of 
learning outcomes, using learning materials provided by AR technology in a way that AR technology affected 
such outcome among dyslexic students in the experimental group. Literature generally found significant ef- 
fects of AR technology on learning outcomes for students (with and without learning disabilities) in many ways, 
enabling them to benefit from its educational contributions (e.g., Bacca et al., 2014; Kellems et al., 2020; Lorenzo 
et al., 2019; Radu, 2014). There are several reasons cited in literature as to the use of AR in learning and they 
include flexibility in instruction designs, support of spatial visualization/audio, enhancement of communication 
skills, enrichment and provision of meaningful learning and transference of knowledge (Cakir & Korkmaz, 2019). 
AR provides interaction with a 3D view of the object from different angles that enhances the students’ skills 
and abilities as well as provides them with the skills to practically apply the object (Cheng & Tsai, 2013; Hsiao & 
Rashvand, 2011). AR technology further provides timely feedback in real-time interaction, leaving the control 
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of learning in the students’ hands (Bujak et al., 2013; Kucuk et al., 2015; Yuen et al., 2011) and AR technology is 
basically a combination between virtual and real world elements established in a single environment through 
images, data and real world contents (Solak & Cakir, 2016). The need for AR technology has been highlighted in 
literature with several studies indicating the requirement of using visual teaching and learning materials and 
contents (multimedia design and computer simulations) to assist the understanding of concepts (Cimer, 2012; 
Mayer, 2009). The use of AR multimedia among children suffering from ASD to enhance their communication 
skills, was examined in Taryadi and Kurniawan’s (2018) study. The authors reached to the conclusion that PECS 
generated a development of communication ability among the children when comparing pre-test and post-test 
scores. In the same way, Lorenzo et al. (2019) examined the effectiveness of AR in enhancing the social skills of 
11 autistic children. Based on the reported results, the students in the experimental group displayed enhanced 
social schools but insignificant differences were revealed. In addition, Kellems et al. (2020) dedicated his recent 
study to investigating the AR effectiveness in teaching mathematics to middle school students that had learn- 
ing disabilities. The students obtained good scores, with improved problem-solving skills after the intervention. 

Moreover, Yalcinkaya (2012) maintained that the development of social skill education in the computer 
environment fitted with web-based distance education system among children with mild mental disabilities. 
The children participated in various activities (e.g., drawing on a drawing tablet, shooting short videos within 
the planned framework, participating in technology-enriched extracurricular activities). Based on the research’s 
results, extracurricular activities driven by technology had a positive influence on the students’ cognitive and 
physical development. Meanwhile, Dogan’s (2015) study on the effects of technology-supported extracurricular 
activities on students that had mental disabilities reached to the conclusion that such activities positively affected 
the students’ cognitive and physical development. Also, Dunleavy et al. (2009) highlighted the potential of AR 
in enhancing learning among students, with its distinct ability to develop immersive hybrid learning environ- 
ments, which facilitated the development of the students’ critical thinking, problem solving and communication 
through collaborative exercises. Furthermore, AR was integrated in communication interventions through the 
connection of elements of augmented and alternative communication and applied behavior analysis strategies 
(STAR) in the research by Almeida et al. (2015). AR also managed to improve the students’ memory and social 
skills. From the above findings in literature, the present study's investigation is justified. 


Conclusions and Suggestions 


The augmented reality (AR) technology has proven suitable for school environment. Promising innovation 
paves the way for teaching and learning opportunities and enhances students’ success. AR technology contributes 
effectively towards achieving optimum results when using AR instruction, one of the best learning methods 
for students with special needs. It supports the inclusion of students and instructors in selecting the right and 
most effective instructional method, to enhance the learning outcomes and to ensure that students have the 
opportunity to express positive attitudes, increased interest, focus, attention and interaction. The natural us- 
age of AR technology in learning is crucial as the AR is a combination of the physical and virtual world which is 
directed towards improving the learning environment through providing more visual and audio information 
through the use of computer and mobile technologies. The combination of real and virtual objects, AR enables 
the visualization of abstract concepts and complex spatial relationships and the experiencing of phenomena 
in a way that cannot be experienced in the real world, and thus, reinforcing the learning experience, increas- 
ing understanding, and enhancing student's motivation, participation, and their engagement. Utilizing the AR 
technology in the special needs learning process, students could be better engaged in learning. They could also 
participate in the learning world via their positive attitudes towards science, thus increasing the demand for sci- 
ence and technology-related majors and careers. The findings of the present research provide vital information 
for instructors and academicians intending to move towards effectively implementing AR technology in their 
learning process for students with special needs. Although, previous research dedicated to AR as an evidenced- 
based approach for instructing students with special needs are still few and far between, and as such, in this 
work, AR instruction is tested using 6" grade students in their science curriculum. The present research’s exami- 
nation is focused on whether grade 6th students with special needs could determine an instruction method in 
advance that would lead to the most optimum learning, therefore, more research is needed in other students’ 
ages and courses. The research’s sample does not promote generalization of results to the general population 
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of students with special needs, as the research results are based on the participating students in the study. 
Hence, the sample of the research should be extended to other students. Regardless of the enhanced evidence 
of the mean scores of the students, the research experiment spanned only 4 weeks and thus, a longer period 
of research is needed for conclusive results. The research is also limited in its data collection method, which is 
the self-report measures, as this is characterized by inflated biases, influenced by the existing social desirability. 
Thus, this research recommends that future authors investigate the research objectives through the use of mixed 
method approach (quantitative and qualitative methods). 
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Introduction 


Healthy eating is a major behavioral factor for dealing with obesity and 
non-communicable diseases such as some types of cancer, cardiovascular 
disease, and type-2 diabetes (World Health Organization; WHO, 2016) ac- 
counting for 71% of the worldwide mortality rate (WHO, 2018). In addition, 
the number of people who undernourished increased from 784 million to 
821 million between 2015 and 2017 (UN, 2019). Thus, the implementation 
of a healthy eating should be encouraged. WHO (2016) expressed that 
a healthy eating should include small amounts of saturated fat, salt and 
refined carbohydrates, as well as high consumption of fruits, vegetables 
and whole grains. Recent data in Turkey have indicated that people do not 
care about their healthy eating behaviors. For example, the proportion of 
obese individuals was 15.2% in 2008, while it increased to 19.6% and 21.1% 
in 2016 and 2019, respectively (Turkish Statistical Institute, 2020). 

Education of individuals is the best solution for increasing levels of 
problems related to healthy eating around the world since health-related 
topics such as healthy eating are involved in socio-scientific issues which 
present great opportunities for the next generations (Fensham, 2012) and 
has a powerful relationship with education (Kickbusch, 2001). Considering 
the importance of health education in socio-scientific issues, in particular, 
science teachers play an essential role since students are taught to decide 
upon related to current social issues (Zeidler et al., 2009). In addition, sci- 
ence teachers can be helpful by teaching basic beliefs and personal values 
as well as decision-making processes and by teaching the science to act 
health behaviors (Zeyer & Dillon, 2014). Moreover, the topics related to 
healthy eating may easily be integrated into science courses (Arnold, 2018) 
and taught by science teachers. However, to organize effective learning en- 
vironments, understanding science teachers’ healthy eating behaviors and 
its antecedents play an important role to help researchers and professionals 
to design influential and appropriate intervention strategies and change 
un-healthy eating behaviors (McEachan et al., 2011). Moreover, teachers’ 
beliefs about healthy eating may affect the likelihood of providing health 


https://doi.org/10.33225/jbse/21.20.573 


JOURNAL 
OF®BALTIC 
SCIEN CE 
EDUCATION 


ISSN 1648-3898 reins 
ISSN 2538-71 38 joni 


Abstract. This study was conducted to 
provide a comprehensive understanding of 
determinants of science teachers’ healthy 
eating intentions and behaviors by combin- 
ing the health belief model and the theory 
of planned behavior into one conceptual 
framework and considering the mediat- 

ing impact of attitude and intention on 
behavior. This study was conducted based 
on cross sectional study design between 
November 2019 and February 2020. A 

total of 13 hypotheses were tested and 
data collected from 563 science teachers 

in Turkey were analyzed using structural 
equation modeling. The results of the study 
showed that the proposed model explained 
the variance in intention and behavior 

at amore satisfactory level than existing 
theories. The results also revealed that all of 
the hypotheses were supported. In addition, 
the mediating role of attitude and intention 
in understanding science teachers’ healthy 
eating behaviors was identified. The study 
can provide important implications for edu- 
cation stakeholders, curriculum developers 
and science educators. 
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education or promoting healthy eating practices in the classroom and may help adjust nutrition education services 
and professional development to meet teachers’ needs (Jones & Zidenberg-Cherr, 2015). Therefore, new stud- 
ies on science education need to be examined factors affecting health behavior and allowing for the inference 
of useful interventions to encourage health literacy and health behaviors in science education (Arnold, 2018). 

There are several behavioral theories understanding health-related behaviors such as the Health Belief 
Model (HBM), Theory of Planned Behavior (TPB), the Health Action Process Approach and the Framework Model of 
Health Literacy. Among them, HBM and TPB are the most used theories for explaining individuals’ healthy eating 
behaviors (e.g., Ates, 2019; Fila & Smith, 2006). Accordingly, the current study used HBM and TPB to understand 
science teachers’ healthy eating intentions and behaviors. The most importance of TPB is that people’ behavioral 
intentions are the determinants that best explain their behaviors (Ajzen, 1991). In the TPB, volitional and non- 
volitional dimensions constitute the basis of intention and behavior (Ajzen & Fishbein, 1980), while the basic 
premises of the HBM are threat perceptions and behavioral evaluations (Bylund et al., 2011). The proficiency of 
these rational theories has been tested to determine healthy eating behaviors (e.g., Kim et al., 2012; Shimazaki et 
al., 2017; Sun et al., 2006) and each of these theories has been shown to be useful in understanding individuals’ 
health related decision-making process (Huang et al., 2020). However, the adequacy of these theories has often 
been doubted (Gerend & Shepherd 2012). Moreover, a great majority of limited number of earlier studies tested 
the direct effect on healthy eating behaviors, while only a few studies investigated indirect effect between con- 
structs (e.g., Deshpande et al., 2009). Considering the insufficient number of studies, therefore, earlier researchers 
called for health-related studies that testing of these theories should continue to better understand the factors 
on healthy eating behavior (e.g., Riebl et al., 2015). It was also suggested that they converge and compare with 
each other to contribute more to the literature, since both the HBM and TPB alone do not completely account 
for complexity and multi-dimensionality of behaviors (Noar & Zimmerman, 2005). Given the lack of empirical 
studies that focus on combining HBM and TPB, it is needed to combine the two theories to determine particular 
constructs that impact specific behaviors, which will help to develop our understandings (Gerend & Shepherd 
2012). Further, to the best of our knowledge, no study has investigated science teachers’ healthy eating inten- 
tions and behaviors by combining two theories and proposing a theoretical conceptual framework. 

To address this literature gap, the current study aimed to: a) propose a conceptual framework for under- 
standing science teachers’ healthy eating intentions and behaviors through two theories (i.e., HBM and TPB); b) 
determine the relative importance of the proposed model compared to the HBM and TPB; c) explore the rela- 
tive importance of the constructs of HBM and TPB within the proposed model to understand the intention and 
behavior; d) examine the mediating role of attitude and intention on healthy eating behaviors. 


Literature Review and Hypothesis Development 
Science Education and Health Education 


Health education has been increasingly significant context for science education (Zeyer & Dillon, 2014) and 
accepted as fundamental dimension of scientific literacy (Zeyer & Kyburz-Graber, 2012) since health is intimately 
connected with human lives (Cruz, 2009). In addition, health topics occupy an important place in science edu- 
cation, as scientific facts and principles are very important in understanding biological systems (Arnold, 2020). 
Moreover, science educators advocate that the necessary education should be given at the school in order to 
create awareness in students against misleading information about health-related behaviors such as healthy 
eating (Fine et al., 2013). However, experts in the field stated that health issues are neglected in science education 
(e.g., Harrison, 2005; Zeyer & Dillon, 2014). For example, in US, there is no prepared health curriculum that can 
be used nationwide, and, in this vein, health is presented as a separate topic, regardless of science and health 
education emphasizes basic subjects about health issues (Keselman et al., 2012). However, in recent years there 
are some initiatives related to including health to science education. Attempts have been made to link health 
issues with science education, socio scientific issue and decision-making process in science curricula in several 
countries (e.g., the Australian Curriculum for Science in Australia (ACARA), 2013; Turkish Middle School Science 
Curriculum, 2018). In addition, in the last decade, some attempts have been done to explain the importance of 
this relationship (e.g., Arnold, 2018, 2020; Zeyer & Dillon, 2014). For example, a book called “Science | Environ- 
ment | Health: Towards a Renewed Pedagogy” was published in 2012 to provide well-grounded perspectives 
on how science education can take advantage of the challenges of health education (Zeyer & Kyburz-Graber, 
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2012). In the light of the information obtained from this book, the relationship between health education and 
science education can be summarized as indicated below. 

a) Health issues can help strengthen students’ interest and motivation for science education. 

b) Health education and science education can encourage informed citizenship and well-informed 

personal choice regarding health. 

c) Incorporating health issues into science education can help promote scientific literacy. 

d) Health education and science education play important role in teaching socio-scientific issues. 

e) Health and science literacy both are inherently knowledge-based. 


Merging of HBM and TPB and Hypotheses Development 


As stated earlier, unhealthy eating and poor dietary habits such as non-consumption of vegetables and 
fruits, skipping breakfast, excessive weight gain, and consuming a lot of fast food (Laska et al., 2012) would 
conclude negative effect on the health of individuals in the short term or long term. Therefore, paying atten- 
tion to individuals’ eating behaviors can be considered vital to living a healthy life in terms of pro-self and pro- 
social concerns, since understanding the determinants of eating behaviors is regarded as an important area of 
research that reduces both the individual and the society. Accordingly, many researchers used the TPB and HBM 
to understand individuals’ healthy eating behaviors. The TPB and HBM are based on expectancy-value frame- 
work (Brewer & Rimer, 2008) and suppose that healthy eating behaviors are related to deliberative and rational 
process in consequence of personal utility and costs (Brewer & Rimer, 2008; Conner & Sparks, 2005). Therefore, 
both theories focus on voluntary and non-voluntary processes, which are accepted as basic factors of rational 
choice theories in explaining healthy eating behaviors. However, TPB assumes that intention is influenced by 
attitudes, subjective norms, and perceived behavioral control (PBC), while HBM embody conflicting values, 
perceptions, and social interactions (Wheeler, 2008) and emphasize the importance of the influence of health 
beliefs on behavior (Janz & Becker 1984). However, both TPB and HBM contain overlapping variables and stress 
the importance of different types of beliefs in understanding human behavior. Thus, it is important to merge the 
two theories to determine certain variables that affect certain specific that will develop understanding related 
to risk prevention behaviors (Gerend & Shepherd 2012). Further, combining constructs of TPB and HBM may 
provide a better accountability to understand healthy eating intentions and behaviors than research that simply 
adopted the model or framework and may develop the explanatory power of the combined proposed model. 
For example, Gerend and Shepherd (2012) revealed that when tested separately, HBM and TPB accounted for 
26 and 39 % of the variance in health-related behaviors, respectively, while merging HBM and TPB increased the 
explanatory power of the merged model by 4%. Accordingly, as often suggested in past studies (e.g., Gerend 
& Shepherd, 2012; Huang et al., 2020), this study postulated a proposed model that a merging TPB and NAM in 
one theoretical model can best explain healthy eating behaviors. 


Health Belief Model 


For nearly 70 years, the HBM has extensively used conceptual model as a guiding theoretical model for 
health-related behavioral interventions (Champion & Skinner, 2008). The HBM was developed originally in the 
1950s to understand whether people attend preventive programs to prevent and determine diseases (Becker 
& Maiman, 1975). 

The initial theory posits that health-related decision-making process is based on primary concepts including 
threat perceptions (perceived susceptibility and a perceived severity), behavioral evaluations (perceived ben- 
efits and perceived barriers) and cues to action (Bylund et al., 2011). Among the threat perceptions, perceived 
susceptibility is people's beliefs about the “likelihood of getting a disease or condition” and perceived severity is 
“feelings about the seriousness of contracting an illness or of leaving it untreated include evaluations of both medi- 
cal and clinical consequences (for example, death, disability, and pain) and possible social consequences (such as 
effects of the conditions on work, family life, and social relations” (Champion & Skinner, 2008, p. 47). Furthermore, 
perceived benefits, one of the behavioral evaluations, can be defined as “potential advantages of engaging in 
the health behavior, including the behavior's perceived efficacy in preventing the undesired outcome” (Gerend & 
Shepherd, 2012, p.172) and perceived barriers are beliefs people have about “the difficulties or hindrances as- 
sociated with a target behavior" (Orji et al., 2012, p.15). One of the other components of HBM is cues to action 
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which are specific triggers that generate an individuals’ sense of need for action (Champion & Skinner, 2008). 
Later, Becker and Rosenstock (1987) added the concept of self-efficacy, which is derived from social cognitive 
theory and defined as “the conviction that one can successfully execute the behavior required to produce the out- 
comes” (Bandura, 1997, p.204). In the HBM, self-efficacy reflects an individuals’ reliance in her/his ability to act 
the health behavior (Weinstein, 1993). The predictive power of HBM has been strengthened with the inclusion 
of self-efficacy (Buglar et al., 2010). 

Earlier empirical studies showed that the HBM is the most relevant explanatory theoretical model when 
examining the motivations for engaging in health behavior (Urbanovich & Bevan, 2020) and widely used to 
understand various health related behaviors, such as healthy eating behaviors (e.g., Deshpande et al., 2009), 
organic food consumption (e.g., Yazdanpanah et al., 2015), healthy risk preventative behavior (e.g., Huang et al., 
2020) and human papillomavirus vaccine uptake (e.g., Gerend & Shepherd 2012). Among them, determinants of 
healthy eating behaviors have been tested by a few of previous studies. These studies revealed the importance 
of the constructs of HBM including perceived susceptibility, perceived severity, perceived benefits, perceived 
barriers, cues to action and self-efficacy on healthy eating intentions and behaviors. For example, Kim et al. 
(2012) found that college students’ perceived severity, perceived benefit and perceived susceptibility were 
positively related to healthy eating intention, while a negative relationship was found between perceived bar- 
rier and healthy eating intention. In another study, Deshpande et al. (2009) found that healthy eating intentions 
of college students were predicted as positive by self-efficacy and as negative by perceived barriers. However, 
it was reported that perceived benefit did not have a significant influence on likelihood to eating healthy. Ina 
study conducted with adult consumers, Cook (2018) found that among the constructs of HBM, perceived barrier 
was negatively related to healthy eating intention. In a study conducted with adult consumers, Orji et al. (2012) 
found that all the constructs of HBM were positively associated with healthy eating behavior except perceived 
barrier which was the only construct that affects healthy behaviors negatively. 

Accordingly, a limited number of past studies confirmed that the constructs of HBM model played an 
important role to explain healthy eating intentions and behaviors. These studies reported that only perceived 
barriers are negatively associated with healthy eating behaviors since barriers such as knowledge, time and 
resource cannot be controlled by individuals and therefore may affect negatively behaviors (Gao et al., 2017). 
In other words, if individuals have higher control over healthy eating behaviors, their intentions to perform the 
behaviors will be stronger. Similarly, if a person feels easy and has relevant knowledge level and skills to eat 
healthily, he/she will probably intend to eat healthily. In addition, past studies reported that in case appropriate 
beliefs including that perceived susceptibility, severity and benefit are held, it triggers healthy eating behaviors 
(Vassallo et al., 2009) and different types of cues such as media campaign and social influence have an impact on 
healthy eating behaviors (Orji et al., 2012). However, the results of earlier studies conducted in different contexts 
may be less generalizable to the participants of the study, namely science teachers. Therefore, it is thought that 
the current study will make important contributions to the literature since there are a limited number of studies 
and the research aimed to study with individuals who have different cultural and educational levels. Based on 
the arguments, then we propose following hypotheses: 

H1: Perceived susceptibility is positively related to healthy eating intentions. 

H2: Perceived severity is positively related to healthy eating intentions. 

H3: Perceived benefit is positively related to healthy eating intentions. 

H4: Cues to action is positively related to healthy eating intentions. 

H5: Perceived barrier is negatively related to healthy eating intentions. 

H6: Self-efficacy is positively related to healthy eating intentions. 


Theory of Planned Behavior 


The theory of planned behavior (TPB; Ajzen, 1991) is extension of the theory of reasoned action (Ajzen & 
Fishbein, 1980) by adding anew construct called perceived behavioral control (PBC) which is non-volitional factor 
(Ajzen, 1991). The theory was proposed to reveal factors affecting various types of human behaviors (Conner & 
Armitage, 1998). According to TPB, the best predictor of human behavior is intention which is considered the 
motivational construct that encourages a person to engage in a certain behavior and intention is affected by 
attitude, subjective norm and PBC (Ajzen, 2005). Attitude refers to positive or negative evaluation of a certain 
behavior implying that the more positive attitudes towards behavior, the more likely it is to carry out this be- 


976 


Wass https://doi.org/10.33225/jbse/21.20.573 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


ISSN 1648-3898 /Print/ DETERMINANTS OF SCIENGE TEACHERS’ HEALTHY EATING BEHAVIORS: COMBINING HEALTH 
BELIEF MODEL AND THEORY OF PLANNED BEHAVIOR 
ISSN 2538-71 38 /Online/ (PP. 5'73-589) 


havior (Ajzen, 2005). Subjective norm defined by Ajzen (1991) as “the perceived social pressure to perform or not 
to perform the behavior" (p. 188) is people’s perceptions related to the views that salient references, including 
family, relative, friend, and colleague, have on their behaviors. PBC is “perceived ease or difficulty of performing 
the behavior and it is assumed to reflect past experience as well as anticipated impediments and obstacles” and it 
is supposed to indicate “past experience as well as anticipated impediments and obstacles” (Ajzen, 1991, p. 188). 

TPB is a widely used parsimonious model and the effectiveness of it in explaining various behaviors has 
been confirmed in a variety of health-related behaviors such as healthy eating behaviors (McEachan et al., 
2011). Its popularity is partly due to its open operationalization with guidelines on how to measure, analyze 
and develop health interventions using theory (Ajzen, 2006). Empirical findings in earlier studies confirmed the 
positive relationships among attitude, subjective norm, PBC indicating the importance of the constructs in the 
TPB in explaining decision-making process of individuals related to healthy eating intentions and behaviors (e.g., 
Ates, 2019). Previous studies have generally focused separately on specific sample groups such as children (e.g., 
Bazillier et al., 2011), adolescents (e.g., Chan et al., 2016; Granhgj et al., 2013), and adults (e.g., Brouwer & Mosack, 
2015). In addition, a majority of them aimed to understand healthy eating intention, while a few studies focused 
on understanding antecedents of healthy eating behavior. For example, Bazillier et al. (2011) tested the essential 
role of children’ (aged 8-9 years old) attitude, subjective norm, and PBC in explaining their healthy eating inten- 
tions. They reported that attitude, social norms and PBC accounted for 35% of the variance in healthy eating 
intention and PBC was the most important predictor of intention. Gronhgj et al. (2013) revealed that perceived 
ease related to healthy eating behavior was the most influential determinant of adolescents’ intentions towards 
healthy eating. In another study conducted by Fila and Smith (2006), the efficacy of the TPB was tested to predict 
healthy eating behaviors of Native American youths aged between 9-18 years old, it was found that subjective 
norm was the most predictive determinant. However, it was reported that there was no significant relationship 
between intention and behavior. Brouwer and Mosack (2015) studying with adult individuals revealed that PBC 
was the most influential factor on healthy eating behaviors. Among the limited number of studies conducted on 
teachers, Ates (2019) studied with various elementary school teachers including Turkish language, mathematics, 
physical education, and classroom teachers and found that PBC was the strongest influence on both intention 
and behavior. Furthermore, positive association was found between healthy eating intentions and behaviors. 

As a result, although past empirical studies contributed significantly to the literature, it is not clear which 
variable has a large impact on intention or behavior and, there are very few studies that specifically examine its 
effect on behavior. Moreover, since the studies are conducted with individuals of very different ages, cultures, 
income, social environment, and professional fields, there is a need to research the healthy eating behaviors of 
science teachers who will raise future generations as science literate and health literate individuals. Therefore, 
the following hypotheses were formulated: 

H7: Attitude is positively related to healthy eating intentions. 

H8: Subjective norm is positively related to healthy eating intentions. 

H9: Perceived behavioral control is positively related to healthy eating intentions. 

H10: Healthy eating intentions are positively related to healthy eating behaviors. 


The Relationship Between Beliefs and Attitude 


Many of contemporary social psychologists argue that cognitive or information processing approaches play 
a major role in forming attitude (Ajzen, 1991). One of the approaches is expectancy-value model developed by 
Fishbein and Ajzen (1975). The model proposed that attitudes develop rationally from beliefs people have about 
the object of the attitude and each belief ties a behavior to a specific result or another trait that occurs through 
the performance of the behavior (Ajzen, 1991). Since the attributes associated with behavior are already evalu- 
ated as positive or negative, we inevitably have an attitude towards the behavior (Ajzen, 2005). 

Similarly, in the context of this study, we believe that health related beliefs including perceived susceptibil- 
ity, perceived severity and perceived benefits affect attitude toward healthy eating and trigger the behavior. 
Perceived susceptibility reflects people's beliefs about the likelihood that a person will experience the outcome 
while perceived severity belief is to what extent people believe a health situation harmful (Orji et al., 2012). 
Accordingly, perceptions of susceptibility and severity can affect attitudes toward the health-related behavior 
(Zhang et al., 2018). Moreover, perceived benefits concern an individual’ belief in targeted behavioral ability to 
decrease the likelihood of being influenced by the health threat (Champion & Skinner, 2008). Thus, people who 
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thought that the particular behavior is safer and more effective maintain a positive attitude toward this behavior 
(Rogers, 2010). Earlier studies confirmed the important role of perceived susceptibility, perceived severity and 
perceived benefits on attitude toward health-related decision-making processes (e.g., Zhang et al., 2018). For 
example, Zhang et al. (2018) reported that health related benefit and risk perceptions were significantly associ- 
ated with attitudes. In another study conducted by Huang et al. (2020) it was found that perceived susceptibility 
and perceived benefit play a fundamental role on attitudes toward health, health-related preventative behaviors. 
However, within our knowledge, no study was conducted examining the influence of perceived susceptibility, 
perceived severity, and perceived benefits on attitude toward healthy eating. Therefore, in this study context, 
we suppose that science teachers who think that they are more likely to be exposed to risk or perceive the se- 
riousness of the risk will have a more favorable attitude towards engaging in healthy eating behaviors. Based 
on the importance of beliefs on attitude as suggested in the TPB and expectancy-value model and earlier study 
findings, following hypotheses indicated in Figure 1 are presented: 

H11: Perceived susceptibility is positively related to attitude toward healthy eating. 

H12: Perceived severity is positively related to attitude toward healthy eating. 

H13: Perceived benefit is positively related to attitude toward healthy eating. 


Figure 1 
The Proposed Model 


Perceived 
Behavioral Control 


Subjective 
Norm 


Perceived 
Susceptibility 


Healthy Eating 
Behavior 


Severity 


Perceived 
Benefits 


Cues to Action 


Perceived 
Barrier 


Research Methodology 


The design of the study was based on the hypothesis that, healthy eating intentions and behaviors of Turk- 
ish teachers could be explained by means of the HBM and the TPB theoretical models and theories. Depending 
upon this above-mentioned hypothesis, the study was conducted using cross sectional study design since data 
are collected from a sample that is from a pre-determined population (Fraenkel et al., 2012). Data of this study 
were collected ona volunteer basis between November 2019 and February 2020. The data analysis was conducted 
through path analysis to examine whether the HBM and the TPB statistically explain teachers’ healthy eating inten- 
tions and behaviors. 
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The participants of the study consist of science teachers in six cities located in different regions of Turkey. In 
the first application, the data were collected from 603 science teachers. However, due to some reasons such as 
items not filled in the questionnaire, multicollinearity and some participants delivering the instrument without 
completing it, 40 of the participants were excluded from the data. Finally, a total of 563 science teachers attended 
the study (age ranged from 22 to 68; M = 39.22, SD = 11.22; MBody Mass Index=25.12). Among the participants, 
53.93% were male and 46.7% were female. Their average teaching experience was 19 years and 18.20% of them 
held a graduate degree. 55.13% of them were married, while 44.87% of them were single. 

Healthy education was provided by science teachers at middle schools in Turkey. In the science curriculum, 
there are some objectives to be reached about healthy eating within the scope of the science lesson. For example, 
in the unit of ‘Our Foods; it is aimed to create awareness in students about the types of food, healthy and balanced 
diet, the damages of smoking and alcohol use, and the benefits of healthy eating (Turkish Ministry of Education 
2018). In addition, pre-service science teachers take healthy eating-based courses such as ‘Nutrition and Health’ 
during undergraduate education (The Higher Education Council of Turkey, 2018). 


Instruments 


Measurement instruments of the study were adopted from earlier studies described in the extant literature. 
Then, the instruments were changed to make them suitable for the current study setting. These instruments have 
been used extensively in many studies of theory expansion and deepening in various contexts, and the validity of 
such tools has been proven many times in these studies (e.g., Astrom & Rise, 2001; Brouwer & Mosack, 2015). The first 
version of the instruments was pre-tested and reviewed by health education and science education academicians. 
Minor revisions were made in accordance with their feedback (e.g., miswriting, spelling error, survey layout). The 
last version of the instruments consisted of demographic information queries and items of the proposed model. 

A total of 11 scales and 40 items including three TPB constructs (attitude; seven items, subjective norm; three 
items, PBC; four items), six HBM constructs (perceived severity; three items, perceived susceptibility; two items, 
perceived barriers; two items, perceived benefits; four items, self-efficacy; four items, and cues to action; three 
items), intention (three items) and behavior (five items) are involved in the study. All the items were rated with 
7-point Likert Type scale from “strongly disagree” to “strongly agree”. Information related to the constructs, items 
and sources is presented in Table 1. 


Table 1 
The Items, Adoption Sources and Data of Convergent Validity 


Construct Item no Statements Source Fare "a AVE CR 
Loading 
psy 4 Herd about myself would change if | ate Becker, 1974: Champion & 705 
Perceived i Skinner, 2008; Rosenstock, ——- 7 51 76 
Severity PSV 2 | am afraid to even think about eating unhealthy 1974; Samoggia & Riedel, 114 . , , 
PSV 3 If | eat unhealthy, my entire life would change 20m 106 
Perceived PS 1 My chances of eating healthy are great Becker, Her Chamiion & 192 
; Skinner, 2008; Rosenstock, 
Suscepti- . ; : 14 64 18 
a 1974; Samoggia & Riedel, 
bility PS2 _Itis likely that | eat healthy 2020 813 
PBR 1 | feel like | am not strong enough to eating Becker, 1974; Champion & 788 
Perceived healthy Skinner, 2008; Rosenstock, 
: : : : 70 59 14 
Barriers pBR2 Eating healthy requires adopting a new habit, 1974; Samoggia & Riedel, 748 
which is difficutt 2020 
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Construct Item no Statements Source Fae : AVE CR 
Loading 
PB 1 | care to look attractive 102 
PB2 I care to have right weight Becker, 1974; Champion & 788 
Perceived - ; Skinner, 2008; Rosenstock, 2 57 84 
Benefits PB3 | believe that eating healthy improves the waymy — 4974; Samoggia & Riedel, 7 . . : 
body looks 2020 , 
PB 4 | believe that eating healthy prevents diseases 144 
SE 1 | feel better when eating healthy 111 
Becker, 1974; Champion & 
Self- SE 2 | usually eat the healthy | choose for myself Skinner, 2008; Rosenstock, 102 
; ; . 76 54 82 
Efficacy SE3 | amable to often eat healthy pee Samoggia & Riedel, 768 
SE 4 | do eat the healthy that | planned 144 
Teacher, Academic Staff or Doctor recommenda- 
scl tions prompted me to eat health ae 
ee y Becker, 1974; Champion & 
Cues to Campaigns (e.g., media: press, TV, and radio) Skinner, 2008; Rosenstock, 
Action One prompted me to eat healthy 1974; Samoggia & Riedel, dae au ke at 
Family members or friends with illnesses eat 
CA3 .703 
prompted me to eat healthy 
ATT 1 Good 179 
ATT2 Useful 74 
ae nie ay Armitage & Conner, 1999; As- tah 
: ATT 4 Pleasant trom & Rise, 2001; Ates, 2019; 122 A 53 89 
bealliy ee Brouwer & Mosack, 2015 
ing is...) ATT5 Enjoyable , 112 
ATT6 Wise .703 
ATT7 Necessary 697 
SN 4 People who are important to me think | should 703 
eat healthy. 
os ; Armitage & Conner, 1999; 
— SN2 pte ae iia tome would approve astro & Rise, 2001; Brouwer 788778 
ae & Mosack, 2015 
SN3 People who are important to me want me to eat 71 
healthy 
PBC 1 | have control over whether or not | eat healthy 191 
Perceived = PBC 2 if want, | can easily eat healthy. Ajzen, 2002: Ates, 2021: Fila 823 
Behavioral ; 19 64 88 
Conifal PBC3 __| think healthy eating is easy for me. & Smith, 2006 812 
PBC 4 Whether or not | eat healthy is mostly up to me. .180 
INT 1 | intend to eat a healthy over the next week Armitage & Conner, 1999; 113 
Intention INT 2 | plan to eat a healthy over the next week Astrom & Rise, 2001; Brouwer .760 70 54 18 
INT 3 | want to eat a healthy over the next week ena entS 141 
BEH 1 | mostly eat healthy foods. 815 
BEH2 _ | eat healthy to keep me from getting diabetes 823 
Behavior BEH3 __ | eat healthy foods when | watch TV. Fila & Smith, 2006 172 Ho J67 91 
BEH4 _ leat fruits. 855 
BEH5 __ | eat vegetables. 837 


Note. “a = Cronbach's Alpha AVE: Average Variance Extracted, CR: Composite Reliability; Negatively worded items were reverse- 


scored.” 
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Data Analysis 


SPSS was used to conduct descriptive statistics. Reliability analysis and exploratory factor analysis (EFA) and 
AMOS was used to perform confirmatory factor analysis (CFA) and path analysis. The data process included two 
stages: Measurement and structural models (Byrne, 2016). Before carrying out the measurement model, the EFA (see 
Table 2) was conducted to identify the factors using principal component analysis (PCA). Firstly, items were tested 
whether they are appropriate to factor analysis. It was found that since Bartlett's test of sphericity was significant 
and Kaiser-Meyer-Olkin value (.902) was higher than .60 (Tabachnick et al., 2018), EFA is suitable for extracting 
salient factors. Finally, PCA revealed that total variance was explained with 79.12%, the eigenvalues were above 
1.0 and factor loading of items in HBM and TPM were more than .50. 


Table 2 
Factor Loadings Obtained from Exploratory Factor Analysis 


Factor component 
Constructs Items 
1 2 3 4 5 6 7 8 9 10 11 
PSV 1 80 
Slo PSV 2 78 
Severity 
PSV 3 18 
Perceived PS 1 80 
Susceptibility PS2 78 
Perceived PBR 1 81 
Barriers PBR 2 78 
PB 1 81 
Perceived PB2 16 
Benefits PB 3 74 
PB4 72 
SE1 15 
_ SE 2 81 
Self-Efficacy 
SE3 oil 
SE4 Sl 
CA1 a7 
Cues to Action CA2 16 
CA3 aA 
ATT 1 aT) 
ATT2 81 
ATT3 19 
Attitude ATT4 14 
ATT5 aif 
ATT6 85 
ATT7 81 
_ SN 1 J7 
Subjective SN2 72 
Norm 
SN3 16 
PBC 1 88 
Perceived PBC 2 79 
Behavioral - 
Control PBC 3 82 
PBC 4 81 
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INT 1 81 
Intention INT 2 82 
INT 3 19 
BEH 1 81 
BEH 2 82 
Behavior BEH 3 1 
BEH 4 19 
BEH 5 80 


Note. PSV= Perceived Severity, PS= Perceived Susceptibility, PBR= Perceived Barriers, PB= Perceived Benefits, SE= Self-Efficacy, 


CA= Cues to Action, ATT= Attitude, SN= Subjective Norm, INT= Intention, BEH=Behavior 


The measurement model consists of reliability including Cronbach Alpha (a) and Composite Reliability (CR) 
and validity including convergent and divergent validity (Bagozzi & Yi, 2012), while structural model tests the 
goodness of fit statistics of the proposed model (Anderson & Gerbing, 1988). The analysis of measurement model 
using maximum likelihood estimation method revealed acceptable goodness of fit indices (y7=416.15, df=145; 
p< .05; 7?/df=2.87; GFl=.90 TFI=.92; CFl=.91; RMSEA=.07; SRMR= .06). In addition, since the value of factor loadings 
and Average Variance Extracted (AVE) were higher than .05, reliabilities of constructs, and CR were above than .70 
(Bagozzi & Yi, 2012) and the square root of the AVE was bigger than correlation values (Carmines & Zeller, 1979), 


convergent (Table 1) and discriminant validity (Table 3) were established. 


Table 3 


Descriptive Values, Correlations and Discriminant Validity 


M sD PSV PS PBR PB SE CA ATT SN PBC INT BEH 

PSV 4.29 .99 TA 

PS 4.19 92 67 80 
PBR 4.08 1.01 -.60 -.67 ll 

PB 4.16 1.18 43 12 -.69 is) 

SE 4.57 1.02 52 MM -.65 62 1B 

CA 4.49 88 39 MM -.54 66 54 12 

ATT 4.85 1.96 60 68 -.67 69 AT .65 13 

SN 4.72 1.14 67 65 -.63 68 67 54 59 Wis) 
PBC 4.88 1.06 38 A2 -45 55 69 i) A2 35 80 

INT 4.66 94 03 14 -.54 66 36 .36 AT 54 42 13 
BEH 4.59 1.12 AQ 69 -4T 60 31 32 A AQ 35 68 82 


Note. The bold values are indicator of square root of AVE for each variable. PSV= Perceived Severity, PS= Perceived Susceptibil- 
ity, PBR= Perceived Barriers, PB= Perceived Benefits, SE= Self-Efficacy, CA= Cues to Action, ATT= Attitude, SN= Subjective Norm, 
INT= Intention, BEH=Behavior 


Research Results 


Structural Equation Modeling 


Structural equation modeling (SEM) indicated the structured models of HBM, TPB and proposed model sat- 
isfactorily fit the data (See Table 4). The proposed model (v?/df=2.65) had a better fit than the TPB (y?/df=2.85) and 
HBM (y?/df=2.92). In addition, the proposed model had more predictive ability for intention and behavior (R?int=.532; 


R?beh=.483) than TPB (R7int=.511; R?beh=.419) and HBM (R2int=.463; R?beh=.354). 
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Table 4 


Goodness Fit Data for HBM, TPB and Proposed Model 


Goodness Fit Statis- 


DETERMINANTS OF SCIENCE TEACHERS’ 


HEALTHY EATING BEHAVIORS: COMBINING HEALTH 
BELIEF MODEL AND THEORY OF PLANNED BEHAVIOR 


(PP. 573-589) 


tics & R? HBM TPB Proposed Model 
x 557.72 547.20 498.20 
df 191 192 188 
2 df 2.92 2.85 2.65 
CFI .90 92 94 
GFI 91 91 93 
TL 91 93 95 
SRMR 05 04 03 
RMSEA 05 05 04 
R? (Adjusted) 
Intention AG 51 53 
Behavior 20 42 48 


Results of path analysis showed that, among HBM constructs, perceptions of susceptibility (B= .48), severity 
(B= .44), benefit (B= .42), cues to action (B= .31), and self-efficacy (B= .39) had a positive relationship with inten- 
tion to eat healthy. However, the relationship between perception barriers and intention was negative (B= -.32). 
Therefore, Hypotheses 1-6 were supported (p<.05). The association among the TPB constructs demonstrated that 
attitude toward healthy eating (B= .35), subjective norm (B= .28) and PBC (B= .49) had a significant influence on 
intention. Finally, there was a significant relationship between healthy eating intention and behavior (B= .40). 
Hence, hypotheses 7, 8, 9 and 10 were all supported. In addition, the path between attitude toward healthy eat- 
ing and perceptions of susceptibility (B= .38), severity (B= .36) and benefit (B= .47) was significant. These findings 
supported Hypotheses 11-13 (p<.05). With regards to total variance explained, it was revealed that 34% of vari- 
ance in attitude was explained by perceived susceptibility, perceived severity, and perceived benefit. In addition, 
constructs of HBM and TPB explained 53% of the variance in healthy eating intention. Lastly, 48% of the variance in 
healthy eating behavior was explained by intention. A summary of the results is displayed in Table 5 and Figure 2. 


Table 5 


SEM Results of the Conceptual Proposed Model 


Hypothesis Pathway Standardized estimate (8) t-value Hypothesis situation 
H1 PS — INT 48 10.689 Supported 
H2 PSV — INT 44 10.158 Supported 
H3 PB — INT 42 9.874 Supported 
H4 CA — INT 31 7.358 Supported 
H5 PBR — INT -.32 7.598 Supported 
H6 SE —> INT 39 9.125 Supported 
H7 ATT — — INT 35 8.025 Supported 
H8 SN — INT 29 6.899 Supported 
H9 PBC — INT AQ 10.789 Supported 
H10 INT —  BEH 40 9.598 Supported 
H11 PS — ~~ ATT 38 8.741 Supported 
H12 PSV — ATT 36 8.301 Supported 
H13 PB — = ATT AT 10.489 Supported 
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Figure 2 


Result of the Structural Equation Modeling 


Behavioral Control 
Subjective 
Norm 


R= 34 


Healthy Eating 
Behavior 


Perceived B 
Susceptibility | |»7% 


Perceived 
Seventy 


* 
pa 


5 
Perceived 
Barrier 


Perceived 
Benefits 


Cues to Action 


Note: *Significant at p< .05 


Testing the Indirect Effects 


The results of the indirect relationship between the constructs in the proposed model showed that perceived 
susceptibility (B= .29, p < .01), perceived severity (B= .26, p < .01), and perceived benefit (B= .24, p < .01) had sig- 
nificant indirect influence on healthy eating intention through attitude. In addition, healthy eating behaviors were 
indirectly influenced by both constructs of HBM (8PS=.23, p<.01; BPSV=.21, p<.01; BPB=.18, p<.05; BCA=.11, p<.05; 
BSE=.17, p<.05, BPBR= -.13, p<.05) and TPB (BATT=.15, p<.05; BSN=.10, p<.05; BPBC=.24, p<.01) via intention. More 
detail related to the indirect relationship is involved in Table 6. 


Table 6 
Indirect Relationships 


On 
Indirect effect of 
Intention Behavior 
Perceived Susceptibility 29" 223" 
Perceived Severity .26* 21* 
Perceived Benefits .24* 18** 
Cues to Action - Ad 
Self-Efficacy - 10" 
Perceived Barriers - -.13 
Attitude - 16 
Subjective Norm - AQ™ 
PBC - .24* 


Note. *Significant at .01, ** Significant at .05 
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Discussion 


In the current study, a comprehensive theoretical framework was prepared based on HBM and TPB to provide 
an understanding of science teachers’ healthy eating intentions and behaviors. These two theories were combined 
into a model by considering the interrelation among their main constructs. The proposed model was supported 
by the data as a comprehensive model of science teachers’ healthy eating intentions and behaviors, and this had 
important implications for demonstrating how pro-social and rational antecedents drive such a health-related 
decision. The results of the study revealed that the proposed model (R?int=.53; R?beh=.48) had better explanatory 
power to explain intention and behavior than TPB (R7int=.51; R?beh=.42) and HBM (R7int=.46; R?beh=.35). This caused 
to be an increase in intention and behavior (2 % and 6%, respectively) in variance accounted for by constructs of 
HBM over and above that accounted for by constructs of TPB. Moreover, 34% of attitude was explained by perceived 
susceptibility, perceived severity, and perceived benefit. Therefore, the findings showed that the conceptual model 
is wide-ranging, adequate, effective and functional in understanding science teachers’ healthy eating intentions 
and behaviors. The comprehensive model can also be an important tool for a clear understanding of science teach- 
ers’ complex decision formation regarding healthy eating in science, health, and nutrition education literature. 

Regarding the relative criticality of used constructs in the model, the prominent importance of PBC was 
identified. Particularly, the non-volitional construct together with other constructs had a significant influence on 
healthy eating intentions and behaviors, and this construct was found to be the most effective when compared 
with other constructs in the proposed model. The finding is in line with the earlier study results that emphasized 
the importance of perceived ease or difficulty of performing the healthy eating behavior (e.g., Ates, 2019; Bazillier 
et al., 2011; Brouwer & Mosack, 2015; Gronhgj et al., 2013). Moreover, the salient role of perceived barrier and self- 
efficacy variables, which were frequently associated with PBC and were emphasized to be significantly related to 
health-related behaviors in previous studies (e.g., Cook, 2018; Gerend & Shepherd 2012; Huang et al., 2020) was 
proven in this study. It implies that the perceived difficulties or hindrances related to the target behavior and the 
confidences in their ability to engage in the behaviors towards different barriers play an essential role on science 
teachers’ healthy eating behavior. In addition, the main results demonstrated that perceived susceptibility, perceived 
severity, and perceived benefit and cues to action predicted healthy eating intentions. The findings implied that 
healthy eating intentions were explained by people's beliefs about the seriousness of the results of healthy eating 
behaviors, potential advantages of displaying healthy eating behaviors, and triggers of healthy eating behaviors 
including teacher, doctor, family, and friend. Such similar findings were obtained to be consistent with the results 
of other researchers (e.g., Kim et al., 2012; Orji et al., 2012). In addition, the current study has an original contribu- 
tion to the literature since it explored the certain antecedent beliefs which were important in affecting science 
teachers’ attitudes toward healthy eating. Examining antecedent beliefs is scarce in earlier study testing the HBM 
and TPB for health-related behavior context (e.g., Huang et al., 2020), and the use of antecedent beliefs has not 
been investigated in the context of healthy eating behavior and science education. Moreover, attitude success- 
fully predicted healthy eating intentions. This indicates the importance of positive attitude towards healthy eating 
among science teachers during behavior. Therefore, a favorable attitude could be a good beginning to motivate 
science teachers’ healthy eating behaviors. Last but not least, science teachers’ intention to eat healthy was also 
determined by their subjective norm implying that behaving in accordance with a healthy eating became a social 
norm. Furthermore, receiving approvals of people who are important to science teachers in Turkey is very impor- 
tant for healthy eating. For example, in the school setting, since other teachers or school administrators are often 
considered important persons to teachers, science teachers are more likely to act according to their approval. This 
result also approves the importance of subjective norm in influencing healthy eating intentions in earlier studies 
(e.g., Ates, 2019; Fila & Smith, 2006; Shimazaki et al., 2017). 

Results of the testing of the indirect influence of constructs of study showed that attitude and healthy eat- 
ing intention played an essential mediating role in the proposed framework. Among the constructs, perceived 
susceptibility, perceived severity, and perceived benefit had indirect impact on intention and behavior through 
attitude toward healthy eating. In addition, all variables of the HBM and TBP were indirectly related to healthy eating 
behaviors. The results were in line with those from earlier studies which is quite rare on health-related decisions, 
which merged the HBM and TBP (e.g., Huang et al., 2020). 

The results have also some practical implications for education stakeholders, curriculum developers and science 
educators. The results can be reported that attitude towards healthy eating behaviors, subjective norm, percep- 
tion of severity, susceptibility, barriers and benefits and beliefs of self-efficacy and cues to action were found to be 


585 


https://doi.org/10.33225/jbse/21.20.573 LTT 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


DETERMINANTS OF SCIENGE TEAGHERS?’ HEALTHY EATING BEHAVIORS: COMBINING HEALTH ISSN 1648-3898  /print/ 
BELIEF MODEL AND THEORY OF PLANNED BEHAVIOR 
(PP. 573-589) ISSN 2538-7138 /Online/ 


significant antecedents in motivating science teachers’ healthy eating intentions. Accordingly, it is essential for the 
ministry of national education, and provincial directorate of national education to design and launch influential 
nutrition-related promotional programs to increase the above-mentioned factors to achieve substantial social 
change towards a healthy eating lifestyle. For example, in-service health promotion training programs designed 
by school management and provincial directorate of national education for science teachers can play an important 
role in developing their beliefs, attitudes, intentions and behaviors. In the education of pre-service science teach- 
ers, who have a very important role in the education of future generations, more courses can be included in the 
curriculum they take at the university, aimed at gaining the awareness of healthy eating. 


Limitations and Future Studies 


Although the study made important contributions to the literature, it has several limitations and sugges- 
tions for future studies. First of all, the study is limited to several data collection tools, hypotheses and educated 
respondents using the convenience sampling method in several cities in Turkey. Therefore, as generation beyond 
the sample in the study is limited, this may be resulted in a demographic bias, may decrease external validity, and 
therefore give rise to sampling bias. Future researchers should extend their studies with more appropriate instru- 
ments and hypotheses using larger sample groups in different cultures. Individuals’ eating beliefs and motivations 
may change according to the cultural characteristics, since individuals are likely to have particular beliefs and needs 
affected by cultural values (Seegebarth et al., 2016). In the study, the data collection tools relied on self-report 
measures rather than the actual behavior, thus the findings should be taken with caution, participants may not 
desire to express their true views due to the social desirability and ethical pressure to indicate their intentions to act 
toward the common good (Kiatkawsin & Han, 2017). Therefore, it is suggested to consider the results in the study 
with this understanding that future researchers can use other data sources such as supervisory ratings (Zhang et 
al., 2017). Finally, since the study focused on constructs of the HBM and TPB, some important constructs can be 
overlooked. For example, the constructs of the proposed model are based on expectancy-value framework de- 
veloped in accordance with rational considerations. Therefore, the proposed model ignores non-rational motives 
in understanding healthy eating behaviors. Therefore, the validity and effectiveness of the theoretical framework 
can be questioned when explaining healthy eating behaviors. In future studies, more theories or models such as 
social cognitive theory, protection motivation theory, health action process approach and model of health literacy 
can be tested to examine the influence of different psychological constructs such as outcome expectancy, value 
of action, motivational and moral considerations on healthy eating intentions and behaviors which can provide 
more comprehensive understanding of the topic. 


Conclusions and Implications 


Decision-making process of individuals regarding healthy eating has not been investigated much in the extant 
literature. This study added several important elements to the existing nutrition, science and health education 
literature. Firstly, this study is probably the first attempt to determine antecedents of science teachers’ healthy 
eating intentions and behaviors combining HBM and TPB in Turkish context. Secondly, the combined model de- 
termined that nine predictors had a significant impact on science teachers’ healthy eating intentions which was a 
strong determinant of healthy eating behavior. Thirdly, attitude toward healthy eating derived from the TPB was 
successfully influenced by perceived susceptibility, perceived severity and perceived benefit, constructs of the HBM. 
Fourthly, attitude toward healthy eating, and intention to eat healthy as mediators were influential in building a 
framework explaining that science teachers make decisions that give importance to healthy eating. Considering 
the successful results of this study, the theoretical and practical importance of the proposed conceptual framework 
including high efficiency, comprehensiveness and applicability is remarkable. 
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Abstract. This study deals with the effects 
of applying drama method in the teach- 
ing of environmental issues in science 
courses on the achievement, retention of 
learning, student interest and attitudes of 
primary school students and also whether 
these effects are dependent on teach- 

ers. The study was designed on the basis 
of the Solomon Four-Group Design. The 
study is a two group pre- and post-test 
experiment in two parts, each part having 
an experimental and control group. The 
courses were delivered to both groups by 
the researcher in the first part and drama 
teacher in the second. The experimental 
groups received 144 hours of teaching for 
six weeks, during which a drama course 
outline was employed, while in the control 
groups the courses followed the course 
outlines covered in the curricula. The data 
were collected using the Environmental 
Achievement Test, Interest Scale towards 
the Environment and Attitude Scale 
towards Science. The findings suggest that 
the teachers had some effects on the meth- 
ods, but the significant differences in scores 
were due to the drama method, which was 
found to improve the achievement of the 
third-grade students and the interest to- 
wards environment and attitudes towards 
science of both third- and fourth-grade 
students. 
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Introduction 


Environmental problems have increased in recent years, suggesting 
that information on the environment is incomplete (Erten, 2006; NEETF, 
2005; Yilmaz, 2006) and human actions are not environmentally benign. 
Therefore, it is important to improve environmental information and ac- 
tivities. Thus, an environmental education program should be developed 
to teach people about the effects of their acts on the environment (WCED, 
1987) and positively change these acts (Ktyict, 2009; Yildirim, 2015). Environ- 
mental issues are a very important subject area for students because of their 
significant effect on their present and future lives. However, environmental 
education is considered only in science courses. Instead, it should be an 
independent part of educational programs (Chepesiuk, 2007) taught using 
active teaching methods, not through information transmission (Gautreau 
& Binns, 2012; Leeming, 1997). 

Studies suggest that, in Turkey environmental education is not con- 
sidered a priority field, that the curricula are not sufficient to improve 
either environmental sensitivity or environmental awareness, and that 
students’ information about interest and sensitivity towards the environ- 
ment are not at a desired level (Erten, 2006; Yilmaz, 2006). Alp et. al. (2006) 
argue that environmental issues are not much more emphasized in Turk- 
ish education programs than other subjects and that the environmental 
education offered is not action-oriented. However, it has been argued that 
environmental education should be in an applied form and should focus the 
students’ immediate environment, suggesting that otherwise it would not 
be meaningful for students (Sobel, 1996, cited in Louv, 2010). Furthermore, 
the class hours devoted to environmental education should be extended 
and teachers should pay greater attention to environmental education. 
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Given that environmental sensitivity begins to develop from the years 9-10, the period of primary educa- 
tion is very important for environmental education (Ktyici, 2009). However, primary school students are still at 
the age of play (Davis, 1998; Willson, 1996) and therefore instructional activities for them cannot be separated 
from games. In this period, playing games is one of the most important needs of children and they could ac- 
quire many significant skills through games. It is well known that, for effective teaching, relevant methods and 
techniques should be applied to encourage activity among students at the age of play (MEB, 2018) and that 
art activities improve students’ achievement (Louv, 2010). When students actively participate in their courses, 
their knowledge becomes permanent and meaningful. Therefore, more student-centred activities should take 
place in the teaching process (MEB, 2018). It is important to organize learning environments in which students 
will actively learn environmental issues through doing and experiencing in primary school (Tilbury, 1994). En- 
vironmental education addresses students’ cognitive, affective, and psychomotor learning areas (Erten, 2006). 
Creative drama is an effective method of improving cognitive, affective, and psychomotor skills in environmental 
education (Alrutz, 2004; Arieli, 2007; Bailey, 1994; David et al., 2013; Kaaland-Wells, 1994; McNaughton, 2004; 
@degaard, 2003; Vargas, 1995). When students learn environmental issues through drama, they may develop a 
connection between daily life and the topics of environmental education and their learning will be longer-lasting, 
being based on using their own observation and experience through games and improvisations. Such learning 
creates an interest in environmental issues in students and improves their achievement (Forgasz, 2013). It also 
improves their attitudes towards the environment. The creative drama method makes learning both enjoyable 
and concrete and therefore makes the topic much more interesting. 

The creative drama method has been used in contemporary teaching at almost all levels of education and 
at every grade level with desirable results on student achievement, student interest, and attitudes in different 
courses and subject matters (Abed, 2016; Arieli, 2007; Bailey, 1994; David et al., 2013; Fleming, Merrel ve Tymms, 
2004, Greenwood, 2001; Kaaland-Wells, 1994; McNaughton, 2004; Vargas, 1995). However, there is stilla need to 
analyse it from different perspectives. in particular, teacher effects are significant in various teaching methods 
and such effects should be determined for the creative drama method (Kaba & Ozdemir, 2012; Ozdemir & Cikla, 
2005). It has been demonstrated in some studies that the realization of the intended achievements in creative 
drama mostly depends on the attitude and character of drama teachers (Kaba & Ozdemir, 2012; Ozdemir & 
Cikla, 2005). It is known that teachers’ inadequacy in the drama method, their inability to plan the process, and 
their negative beliefs about the method adversely affect the implementation. Isenberg and Jalango stated that 
teachers have a strong impact on drama studies with children (cited in Kéksal, 2012). Moreover, the creative 
drama method is highly influenced by teacher characteristics (Kaba & Ozdemir, 2012; Ozdemir & Cikla, 2005). 
More specifically, its success is highly dependent on teachers’ perspectives on the method, their desire to 
employ it, and their competency in using it. Teachers should have information about the drama methods and 
techniques. Most drama teachers have an insufficient knowledgebase (Adiglzel, 2002). If the teachers do not 
have the necessary knowledge and skills for creative drama, the effects of the drama activities will not reach 
the desired level. Teachers should therefore improve their knowledge and skills in creative drama (K6ksal, 2012), 
and the creative drama method should be employed by teachers who know it well. This study is original in that 
it included a sample of third-grade students and dealt with the teacher effect of the creative drama method. 


Research Aim and Research Questions 


Besides the methodology, based on the importance of teacher effect on experimental studies and use of 
a new discipline; creative drama as a methodology, the aim of this research is to reveal the effects of using the 
creative drama method in teaching environmental problems on the science achievement, attitudes towards 
science, environmental interest scores and their retention of primary school third and fourth grade students. 
Another important aim of the research is to reveal whether the practitioner has an effect on the results obtained 
after the experimental procedure. These aims were operationalized through the following research questions: 
1) What is the effect of applying the creative drama method in teaching environmental issues on the achieve- 
ment, science attitude, retention, and environmental interest scores of the third- and fourth-grade students in 
the experimental group? 2) Do the scores of the experimental group students differ according to whether the 
researcher or the teacher is in the classroom?” 
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General Background 


In this study, an experimental method was used. The study was based on the Solomon Four-Group Research 
Design. The research consisted of a combination of two pre-test-post-test control groups. The model was devel- 
oped to test the effect of the teachers implementing the drama method on the experimental process (Figure 2). 


Figure 1 
The Model Employed in the Study 


Study2: 


Testing of the effects of 
Creative Drama through semi- 


experimental model in below 
Study1: Commentary 


G1 RO X O 
11 1.2 The analysis of data 


gathered from semi- 
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24 
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Gl RO xX O 
44 


4.2 


Figure 2 
Groups Taught by the Researcher and Teacher 


Control Groups 


Experimental Groups 


Teacher’s Class 


As can be seen in Figure 2, the courses for the Experimental and Control groups were delivered by the research- 
er-classroom teacher, and those for the Experiment2 and Control2 groups by a drama teacher. The courses were 
taught by different teachers to test the effect of the teachers on the experiment and the students, if any. The goal 
here was to test the effect of the method on the experimental process and the students without the influence the 
teacher makes. Therefore, the teacher effect was controlled. Thus, it can be said that the result in the experiment 
emerged independently from the teacher's influence. To clarify whether the differences in results were the effect 
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of the teacher, the method employed, or both the teacher and the method. Since one of the teachers who taught 
the courses was also the researcher, it was thought that bias of the independent variable might affect the research. 
It may also be that bias in the independent variable may affect the study because one of the teachers was also 
the researcher. Thus, the difference of the teacher could be clearly controlled. The difficulty of the model is that 
any study carried out with four groups simultaneously requires a significant amount of energy and effort from the 
researcher (Fraenkel & Wallen, 1996). Another difficulty specific to this research is that both teachers needed to be 
an expert in drama. Table 1 summarizes the experimental design of the study. 


Table 1 
Experimental Design of the Study 


Group Label Pre-test Activity Practitioner Post-test Cae 
Experiment, EATISE,ASTS O Drama Method Researcher EAT,ISE,ASTS O EATO 

3E-4B 14 1.2 13 
Control, EAT,ISE,ASTS O Non Researcher EAT,ISE,ASTS O EATO 

31-4H 24 22 2.3 
Experiment, EATISE,ASTS O Drama Method Teacher EAT,ISE,ASTS O EAT O 

3C-4E 34 32 3.3 
Control, EAT,ISE,ASTS O Non Teacher EAT, SEASTS O EATO 

3G-4G 44 42 43 

Figure 3 


The Order of the Data Collection Tools 


Before 


implementation 


Control Groups 


Experimental Groups 


Researcher 
Group1 


Teacher 
Group1 


implementation 


Teacher 
Group2 


Researcher 
Group2 


After 
implementation 


The courses given to the Experiment, and Experiment, groups were delivered using the drama course out- 
lines and those to Control1 and Control2 groups using the course outlines given in the education program. The 
administration of the data collection tools is given in Figure 3. 
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Participants 


The participants were third- and fourth-grade primary school students attending a public school in the Mamak 
District of Ankara during the 2015-2016 school year. The participants were selected through random sampling 
(Balct, 2005). More specifically, 231 third- and fourth-grade students who did not take a drama course participated 
in the study. There were four third-grade (3E, 3C, 3G, and 31) and four fourth-grade (4E, 4B, 4G, and 4H) branches. 
Given that the students were similar in terms of age and socio-economic status, there was no special process to 
match the students. The experiment and control groups were selected randomly. Therefore, in the study a semi- 
experimental design was employed. Table 2 shows the distribution of the participants. 


Table 2 
Distribution of the Participants 


Groups Grades Female Male N Total Total 
; 3-E 14 9 23 
Experiment 1 43 
4-B 9 11 20 
89 
: 3-C 18 5 23 
Experiment 2 46 
4-E 12 11 23 
3-| 15 15 30 
Control 1 72 
4-H 28 14 42 
142 
3-G 19 14 33 
Control 2 70 
4-G 20 17 37 
Total 231 


The quantitative data of the study were collected through the administration of the following instruments: 
Environmental Achievement Test (EAT), Interest Scale towards the Environment (ISE), and Attitude Scale towards 
Science (ASTS; Geban et. al., 1994). All these instruments were employed as both pre- and post-test. In addition, 
the EAT was administered to all groups five weeks after the study as a retention test. 


Instrument and Procedures 


Environmental Achievement Test (EAT): The EAT has versions for third- and fourth- grade students. It was developed 
to measure the current knowledge and retention of students. It was used as pre-test, post-test, and retention test in 
the study. The items of the test were based on related studies and textbooks, and then revised. On the basis of expert 
feedback, the test was used in a pilot study with a sample of 489 third- and fourth-grade students. The mean difficul- 
ties for the third-grade and fourth-grade versions were found to be 0.64 and 0.54, respectively. Item discrimination 
was found to range between 0.16 and 0.68. In the interpretation of the discrimination of items, only those items with 
a discrimination greater than 0.30 and those with a discrimination of 0.20-0.29 were included in the test. The others 
were excluded (Atilgan et al., 2009). For the third-grade version, the KR-20 internal consistency coefficient was found 
to be 0.773. As a result, item 2 was excluded, resulting in a 20-item test. For the fourth-grade version, item 11 was 
excluded, yielding a KR-20 internal consistency of 0.827. It consisted of 24 items. 

Interest Scale towards The Environment (ISE): The Interest Scale towards the Environment (ISE) was developed 
by the author. At the beginning of the development of the scale, a group of 40 students were asked to write essays 
on the environment which then were analysed using content analysis. Then the science curricula of the third- and 
fourth- grades and those gains on environment were reviewed. Next the related studies (Erdogan & Marcinkowski, 
2015; Maskan et al., 2005) were reviewed before the draft was developed. To test its understandability and duration, 
a study was carried out on a sample of 50 students. On the basis of expert feedback and the results of the study, a 
second draft version with 50 items was constructed. A pilot study was carried out on a sample of 545 third- and fourth- 
grade students. Exploratory factor analysis was carried out to determine the factor structure and validity of the scale, 
revealing that it had a single factor. In the scale development process, the criterion that the factor load should be at 
least 0.45 was used (BuyUkoztirk, 2012). The test includes 11 items on 5-point Likert scale. Table 4 gives the variance 
values, factor loads, and item-total correlations. 
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Total 


6.43 
1.49 
1.26 
1.01 
0.96 
0.91 
0.87 
0.80 
0.79 
0.72 
0.71 


Variance accounted % 


29.21 
6.76 
5.72 
4.61 
4.37 
4.14 
3.97 
3.65 
3.59 
3.27 
3.21 


Eigenvalues 


Cumulative Variance % 


29.21 
35.97 
41.69 
46.30 
50.67 
54.81 
58.78 
62.43 
66.02 
69.29 
72.50 


Kaiser-Meyer-Olkin sampling adequacy: .920 
Chi-square value of the Bartlett's Test of Sphericity = 2860.930, SD=231, p = .001 


The Kaiser-Meyer-Olkin statistic is .920; and a value greater than .50 indicates that the sample is adequate 
(Kalayci, 2010). The Bartlett’s Test of Sphericity was used to determine whether the data were appropriate for 
factor analysis. The results showed that the data were proper for the analysis (p < .05). It was also found that the 
factor loads varied between .483 and .559 and that the item-total correlations ranged between .404 and .526. The 
Cronbach's a for the scale was .877, indicating a high level of reliability. The eigenvalue for the single factor was 


6.43, accounting for 29% of the variance. Figure 4 shows the eigenvalues. 


Figure 4 
Graph of Factor Eigenvalues 
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Attitude Scale towards Science (ASTS): This scale, developed by Geban et al. (1994), analyses the student at- 
titudes towards science courses. The scale has 15 items and its value of Cronbach's a was found to be .83 in this 
study. It was applied as a pre- and a post-test in this study. 

Creative Drama Course Outlines: The author developed 14 drama course outlines. These outlines were used 
in the courses given to the experiment groups. The outlines were developed on the basis of student interests, 
the developmental characteristics of children, the structure of the method, teachers’ guidebooks, annual lesson 
plans, and various textbooks (Davis & McGregor, 201 1; Ustiindag, 2012). The plans were examined by five classroom 
teachers and three creative drama teachers at the school and modified on the basis of their feedback. The course 
outlines included three sections: preparation-warming, improvisation, and evaluation. During the first section, the 
students prepared for the course. In the improvisation section, the roles were distributed to the students prepared 
for their roles. In evaluation, the students briefly talked about the implementation. A sample course outline is given 
in the Annex 1. 

Pre-implementation Activities: The researcher participated in drama teacher training courses before the study 
to learn about the method. The course plans were examined by five classroom teachers and three creative drama 
teachers at the school. The plan was employed in a course at the fourth-grade level. The experimental subjects 
received six hours of a pre-implementation course designed to test the experiments in terms of aims and duration. 
The findings from these courses were used to improve the course outlines. Before implementation, the researcher 
and teacher gave information about the creative drama method to the students in the experimental group. They 
were also informed about what they were expected to do during the implementation and activities. 

Implementation: After receiving permission from the Ankara National Education Directorate and Governor- 
ship, the study was initiated during the spring semester of the 2015- 2016 school year. Although the study began 
for both groups simultaneously, the experimental section lasted longer. More specifically, the implementation for 
the experimental groups lasted six weeks, and a total of 36 hours, 6 per week, were devoted to the study. The total 
of course hours for the control group was 144. It was limited to 12 hours or four weeks, as covered in the science 
education program. The school’s multi-purpose hall was used as a study area, which was redesigned to facilitate 
the drama activities. However, in some cases the creative drama method was employed in crowded classes and small 
areas. Therefore, in the first two sessions classroom rules to guide the student behaviours were identified and hung 
on the wall of the classroom for reference whenever necessary. In this way, all students were made active participants 
in the process and became more collaborative (Levey, 2005). At all stages, especially in the most important stage of 
creative drama, namely the improvisation phase, every student in the class was given a role. They were also made 
active participants of the drama activities. 

Treatment Verification: The researcher observed both the experimental and control groups during implementa- 
tion in order to monitor the degree of application of the experimental protocols. The observation form created by 
the researcher was used to ensure that the drama lesson plans were applied only in the experimental groups. The 
observation data showed that the drama method was not applied to in the control groups, nor was the traditional 
method employed in the experimental groups. In addition, the observation form was important for comparing 
the atmosphere in the classrooms. On the observation form, there were 30 items on a 4-point Likert scale: “good, 
intermediate, bad, not implemented”. For the observation form the observation form used by Yilmaz (2007) in the 
PhD Thesis entitled Finding Anchoring Analogies to Help Student's Misconceptions in Physics was adopted. The 
Observation Form is given in the Annex 2. 


Data Analysis 


The quantitative data were statistically analysed with the significance level set at .05. The following statistical 
methods were applied in this study: SPSS 22 was used. 
1. The Kolmogorov-Smirnov and Shapiro-Wilks normality tests were used to examine the distribution of 
the data obtained in the study and to test whether the data showed normal distributions. 
2. Independent- samples t-tests and the Mann- Whitney U test were used for pairwise comparison of 
classes EAT, ISE, ASTS and ASTS Retention scores. 
3. MANCOVA was used to compare the EAT post-test scores of the experimental and control groups. 
Validity and Reliability of the Research: The factors that might affect the validity, reliability, and results of the 
research were all determined and controlled. The study group characteristics that might threaten the internal 
validity of the study were described in detail, and it was sought to control the different characteristics of the study 
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groups as independent variables. The participants were randomly assigned to the groups. The matching of the 
groups in terms of socio-economic condition and other variables was made with maximum care. The data collec- 
tion tools were administered in a safe and comfortable environment; each of the scales was administered on a 
different day; and the data analysis was carried out in a proper environment. The fact that both the researcher and 
a drama teacher taught the courses is important for determining any teacher effect observable in the research. The 
research model, study group, data collection tools, data analysis, and interpretation process were all explained in 
detail to establish the external validity of the study. 


Research Results 
Effects of the Method 


The normality test was performed to test the assumption of univariate normality, and the skew values were 
examined. In order to analyse multivariate normality, scatter- plots were investigated. The relationships between 
the control variables and dependent variables were analysed using the Pearson correlation coefficient. In addition, 
the relationships among the dependent variables and those between the dependent variables and others were 
analysed using scatter-plots. To examine the equality of the regression lines, they were analysed by considering 
the significance of EAT pre-test scores on the “Test of Between Subjects” table. 

Third-Grade Students: Before the MANCOVA analysis the normality of the distributions of the control variables 
(EAT, ISE, and ASTS pre-test) and dependent variables (EAT, ISE, and ASTS post-test) on the groups or independent 
variables was analysed. The Shapiro- Wilks and Kolmogorov-Smirnov tests were used to analyse the skewness 
and kurtosis values. The analysis showed that the scores were normally distributed. As stated earlier, multivariate 
normality was diagrammed as shown in Figure 5. 


Figure 5 
Distribution Chart of the Control and Dependent Variables for the Third- Grade Students 
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As can be seen in Figure 5, the control variables and dependent variables show a linear correlation. The 
distributions are expected to be elliptical, and the figures above seem to be nearly elliptical. When the Pearson 
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correlation coefficient of the control and dependent variables of third-year students were examined to analyse 
the relationship between the common and dependent variables, it was found that all variables showed a sig- 
nificant correlation. In addition, ISE pre-test scores were found to have low correlations with the ASTS pre-test 
and EAT post-test scores. The ASTS pre-test scores had low correlations with EAT post-test scores, and similarly, 
EAT post-test scores had low correlations with ISE post-test scores (r < .30). The homogeneity of the variance- 
covariance matrices of the scores of the dependent variables was tested. The results of the Box’s M test showed 
that the variance-covariance matrices were not homogeneous (p < .05). According to the results of Levene’s test, 
conducted to test the equality of variances the variances of the EAT post-test and ISE post-test scores were found 
not to be equal (p < .05), while that of the ASTS post-test scores was found to be equal (p > .05). The assumption 
of the equality of the slopes of the regression lines was tested, and it was found that the common effects of 
the dependent variables by groups were significant (p < .05). In other words, the slopes of the regression lines 
are not equal. According to the results of the analysis, the descriptive statistics of the dependent variables are 
given in Table 4. 


Table 4 
Mean and Adjusted Average Values of Dependent Variables by Groups 


Group Mean Corrected mean N 
Experimental group 12.02 12.26 45 

EAT post-test 
Control group 11.30 11.13 63 
Experimental group 104.67 105.13 45 

ISE post-test 
Control group 99.48 99.14 63 
Experimental group 71.42 71.95 45 

ASTS post-test 

Control group 65.95 65.58 63 


Table 4 shows the EAT, ISE, and ASTS post-test score averages and corrected averages of 108 students in the 
experimental and control groups. The mean scores of the students in the experimental group were higher on all 
three tests. In terms of common effects, the differences in the post-test scores on the EAT LF aici = 4.66, p < .05], ISE 
[F103 = 12-12, p < .05], and ASTS [F., ,,,, = 31.64, p < .05] by group were significant. On the basis of the partial n? 
values, it is seen that the variance accounted for by the post-test scores of the EAT is 4%; by the ISE, 11%; and by the 
ASTS, 24%. The Wilks’ Lambda values, which demonstrate the significance of the effects of common variables and 


group variables, are given in Table 5. 


Table 5 
Wilks’ Lambda values on Common Variables and Group Variables 


Wilks’ Lambda F p n, Observed power 
EAT pre-test val 13.83 .000 29 1.00 
ISE pre-test 87 4.89 .003 AS 90 
ASTS pre-test 69 15.31 .000 31 1.00 
Group 14 11.90 .000 26 1.00 


As can be seen in Table 5, the pre-test scores of the EAT, ISE, and ASTS have basic effects on the group variables 
(p < .01). Partial n? values show that the total variance of the dependent variables accounted for by the EAT pre- 
test scores is 29%; by the ISE pre-test scores, 13%; and by the ASTS pre-test scores, 31%, while the group variable 
accounts for 26%. The observed power values for whether the null hypothesis is incorrectly rejected are .90 and 
higher, indicating that the null hypothesis was correctly rejected. 

The significance of the difference between the averages was examined by group. It was found that the post-test 
scores on the EAT, ISE, and ASTS differ significantly by group (p < .05). The significance of the difference between 
the adjusted mean scores was found to be significant (p < .05). Accordingly, the EAT, ISE, and ASTS post-test mean 
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scores of the students in the experimental group were higher than those of the students in the control group. This 
shows that the experimental procedure is effective. 

Fourth-grade Students: Before the MANCOVA analysis, the normality of the distribution of the control variables 
(EAT, ISE, and ASTS pre-test) and dependent variables (EAT, ISE, and ASTS post-test) on the groups or independent 
variables was analysed. The results of the normality tests (i.e., Shapiro-Wilks and Kolmogorov-Smirnov) indicated 
that the scores had a normal distribution without any significant deviation (Field, 2005; Kalayci, 2010; Tabachnick 
& Fidel, 2001). The scatter plot drawings for multivariate normality analysis were developed. These figures are 
given in Figure 6. 


Figure 6 
Distribution Chart of the Control and Dependent Variables for the Fourth- Grade Students 
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As can be seen in Figure 6, the relationship between control variables and dependent variables is linear. The 
distributions are expected to be elliptical, and the analysis showed that the distributions are indeed close to the el- 
liptical. The Pearson correlation coefficients for the control and dependent variables of the fourth- grade students 
were calculated to investigate the relationship between common and dependent variables, showing that the 
EAT pre-test scores have significant correlations with the ASTS pre-test, EAT post-test and ASTS post-test scores 
(p < .05). In addition, the ISE pre-test scores are significantly correlated with the ASTS pre-test, ISE post-test, and 
ASTS post-test scores (p < .05). Furthermore, the EAT post-test scores are significantly correlated with the ASTS 
post-test scores (p < .05). However, the EAT pre-test scores have no significant correlations with the ISE pre-test 
and post-test scores (p > .05). In addition, the ISE pre-test and post-test scores are not significantly correlated (p 
> .05). Furthermore, the EAT post-test scores have no significant correlation with the ISE post-test scores (p > .05). 

The homogeneity of the variance-covariance matrices of scores related to dependent variables was tested. The 
covariance matrices are not homogeneous according to Box's M test (p < .05). According to the results of Levene's 
test, conducted to test the equivalence of variances, the variances of the EAT, ISE and ASTS post-test scores are 
not equal (p < .05). The assumption of the equality of the slopes of the regression lines was tested, showing that 
the common effects of the dependent variables were significant by group (p < .05). In other words, the slopes of 
the regression lines are not equal. The descriptive statistics of the dependent variables in the MANCOVA analysis 
are given in Table 6. 


599 


https://doi.org/10.33225/jbse/21.20.590 LTT 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


AN EVALUATION OF THE GREATIVE DRAMA METHOD USED IN THIRD- AND FOURTH- GRADE ISSN 1648-3898 print 
CLASSES ON ENVIRONMENTAL TOPICS BY TEAGHING METHOD AND TEAGHER EFFECT 

(PP. 590-611) ISSN 2538-7138 /Online/ 
Table 6 


Mean and Adjusted Average Values of Dependent Variables by Group 


Group Mean Corrected mean N 
Experimental group 15.00 13.19 43 

EAT pre-test 
Control group 12.68 13.67 79 
Experimental group 104.81 104.01 43 

ISE pre-test 
Control group 98.59 99.03 79 
Experimental group 71.53 69.75 43 

ASTS pre-test 

Control group 65.35 66.33 79 


Table 6 shows the mean scores and corrected averages of the EAT, ISE, and ASTS post-test scores for 122 
students in the experimental and control groups. Therefore, the EAT post-test scores do not vary significantly by 
group IF... ,,, =-36, p > 05]. However, the post-test scores of the ISE [F., ,,,, = 6.50, p < .05] and ASTS [F., ,,,, = 7.17, 
p < .05] show significant variance by group. On the basis of the partial n’ values, it can be stated that the post-test 
score of the EAT accounts for 0.3% of the variance; this figure is 5.3% for the ISE post-test and 5.8% for the ASTS 
post-test. As can be seen, the variance accounted for by the EAT post-test is very low. The Wilks’ Lambda values, 
which test the significance of differences in common variables by group, are given in Table 7. 


Table 7 
Wilks’ Lambda Values for Common Variables by Group 


Wilks’ Lambda F p nn Observed power 
EAT pre-test 65 20.26 .0001 35 1.00 
ISE pre-test 93 3.01 033 07 10 
ASTS pre-test 85 6.84 .000 15 97 
Group 92 3.29 023 .08 14 


As can be seen in Table 7, the pre-test scores of the EAT, ISE, and ASTS show a main effect of group (p < .05). 
The partial n? values indicate that the total variance of the dependent variables accounted for is 35% for the EAT 
pre-test scores, 7% for the ISE pre-test scores, 15% for the ASTS pre-test scores, and 8% by group. The observed 
power values for whether the null hypothesis is incorrectly rejected were less than .90 for the ISE pre-test, but 
higher than .90 for the EAT pre-test and ASTS pre-test. Given that this value is .90 or greater, it indicates that the 
hypothesis of absence of effect is correctly rejected. 

The significance of the difference between the mean scores was examined by group. It was found that the 
post-test scores of EAT did not significantly vary by group (p > .05). However, the post-test scores of the ISE and 
ASTS significantly differed based on groups (p < .05). A significant difference between the adjusted mean scores 
(p < .05) was found. Therefore, it can be argued that the ISE and ASTS post-test mean scores of the students in the 
experimental group were higher than the students’ mean scores in the control group. Hence, this shows that the 
experimental procedure was effective. 

The pre-test and post-test scores of the third- and fourth- grade students were compared. It was found that 
for the third- grade students the post-test scores of the EAT, ISE, and ASTS differ significantly in favour of the ex- 
perimental group. For the fourth- grade students, the post-test scores of the ISE and ASTS differ significantly in 
favour of the experimental group. It may be stated that the use of the drama method in teaching environmental 
topics improved the academic achievement of the third- grade students and the student interest and attitudes of 
both third- and fourth- grade students. 
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The Teacher Effect 


The groups were then compared to answer the following research question: Do the scores of the experimental 
group students differ according to whether the researcher or the teacher is in the classroom? The teacher effect 
was analysed on the basis of the results of the EAT, ISE, and ASTS and on the results of the ASTS as a retention test 
by a comparison of the Experiment, and Experiment, groups whose lessons the researcher and the teacher led, 
respectively, for a significant in results. The t-test was used to analyse the significance of the differences of the gains 
in EAT scores of the Experiment, and Experiment, groups of third- grade students, while the U test was used to 
analyse the significance of the differences of the gains in EAT scores of the Experiment, and Experiment, groups 
of fourth- grade students. The results of the t-test and U test are given in Table 8. 


Table 8 
FTestand U-Test Results for the Gain in EAT Scores of Third- and Fourth- Grade Students by Group (Experiment, and Experiment.) 


Group N X SD df t p 
Experiment, 23 65 2.29 44 -1.17 248 
Third grade , 
Experiment, 23 1.52 2.78 
Group N Mean Rank Rank Sum U p 
Experiment, 20 13.43 268.50 58.50 .0001 
Fourth grade 
Experiment, 23 29.46 677.50 


As Table 8 shows, the gains in the scores of the third- grade students in the Experiment, and Experiment, 
groups did not differ significantly [t,,, = -1.17, p > .05]. In other words, they obtained similar gains in scores. The 
experimental process similarly influenced the students of both groups. However, the gains in scores of the fourth- 
grade students in the Experiment, and Experiment, groups significantly differed (U = 58.50, p < .05). More specifi- 
cally, on the basis of the mean ranks, the achievement of the students in Experiment, was higher. These findings 
indicate that there was no teacher effect for the third- grade students for either the experimental or the control group. 
However, this effect was observed for fourth- grade students. The reason for this finding can be stated as follows: 
Both teachers are proficient in drama, and the drama teacher is more experienced than the researcher in the drama 
method. The t-test was used to analyse the significance of the differences in gains in the EAT scores of the Control, 
and Control, groups of third- grade students. The U test was used to analyse the significance of the differences of 
the gains in EAT scores of Control, and Control, groups for the fourth- grade students. The results of the t-test and 
U test are given in Table 9. 


Table 9 
T-Test and U-Test Results of Gains in EAT Scores of Third- and Fourth- Grade Students by Group (Control, and Control.) 


Group N X SD df t p 

Control, 30 A7 2.98 61 1.28 .205 
Third grade 

Control, 33 -.88 3.45 

Group N Mean Rank Rank Sum U p 

Control, 42 45.57 1914.00 543.00 019 
Fourth grade 

Control, 37 33.68 1246.00 


Table 9 indicates that the EAT gain scores of the control groups (Control, and Control.) do not vary significantly 
by group [t,,, = 1.28, p >.05] among the third- grade students. More specifically, those in Control, had similar gains 
in EAT scores as those in Control,. However, among the fourth- grade students, the EAT gain scores of the control 
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subjects (Control, and Control.) differ significantly by group (U = 543.00, p < .05). The mean ranks show that the 
academic achievement of the fourth- grade students in Control, was higher than that in the other control group. 
These findings indicate that there was no teacher effect in the third- grade students regardless of the group. How- 
ever, there was a teacher effect for the fourth- grade control groups. The reason for that can be the fact that both 
teachers had a competency in delivering the courses using the traditional teaching method. The Mann- Whitney 
U test was used to analyse the significance of the differences in the gains in ISE scores of the Experiment, and 
Experiment, groups of third- grade students. The U test was used to analyse the significance of the differences of 
the ISE gain scores of Experiment, and Experiment, groups of fourth- grade students. The results of the U test are 
given in Table 10. 


Table 10 
U-Test Results of the Gains in ISE Scores of Third- and Fourth- Grade Students by Groups (Experiment, and Experiment.) 


Group N Mean rank Mean sum U p 
Experiment, 22 17.68 389.00 
Third grade 136.00 .008 
Experiment, 23 28.09 646.00 
Experiment, 20 28.08 561.50 
Fourth grade 108.50 .003 
Experiment, 23 16.72 384.50 


As can be seen in Table 10, the gains in ISE scores of third- grade students (Experiment, and Experiment.) 
showed significant differences by group (U = 136.00, p < .05). The mean rank suggests that the interest of the 
students in Experiment, is much higher. The gains in ISE scores of fourth- grade students (Experiment, and Experi- 
ment,) also showed significant differences by group (U = 108.50, p < .05). The mean rank suggests that the interest 
of the students in Experiment, is much higher. Therefore, it can be argued that there is a teacher effect for both 
experimental groups of third- and fourth- grade students. T-tests were used to analyse the significance of the dif- 
ference in the gains in the ISE scores of the Control, and Control, groups of the third- and fourth- grade students. 
The results of the t-tests are given in Table 11. 


Table 11 
Results of T-Test of the Gains in ISE Scores of Third- and Fourth- Grade Students by Group (Control, and Control.) 


Group N X SD df t p 
Control, 30 1.73 17.67 

Third grade 35.26 52 607 
Control, 33 -.30 6.11 
Control, 42 05 17.50 

Fourth grade 56.87 -.53 585 
Control, 37 1.68 7.45 


As can be seen in Table 11, the gains in ISE scores of the third- grade control subjects significantly differed by 
group (Control, and Control.,) [t,,, = .52, p >.05]. Both groups showed much the same gains in ISE scores. The gains 
in ISE scores of the fourth- grade control subjects also did not differ significantly by group (Control, and Control.) 
[ts6¢7) = —-23, P >.05]. More specifically, the fourth- grade students in the Control, had higher gains in scores than 
those in Control.,. In terms of the control group, there was no significant difference between the third- grade and 
fourth- grade students. A t-test was used to analyse the significance of the differences in the gains in ASTS scores 
of the Experiment, and Experiment, groups of third- and fourth- grade students. The results of the t-tests are given 
in Table 12. 
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Table 12 


Results of T-Test of the Gains in ASTS Scores of Third- and Fourth- Grade Students by Group (Experiment, and Experiment.) 


Group N X SD df t p 
Experiment, 23 7.39 5.40 
Third grade 44 - 43 670 
Experiment, 23 8.39 9.80 
Experiment, 20 740 5.17 
Fourth grade 41 5.10 .001 
Experiment, 23 -.13 4.51 


Table 12 indicates that the gains in ASTS scores did not significantly differ for the experimental groups 
(Experiment, and Experiment.) of the third- grade [t.,, = —.43, p > .05], suggesting that the experimental process 
had similar effects on both experiment groups. The gains in ASTS scores did, however, significantly differ for the 
experimental groups (Experiment, and Experiment.) of the fourth grade [t.,, = 5.10, p < .05]. More specifically, the 
students in Experiment, had much higher gains in scores on the ASTS than did students in Experiment, suggesting 
that the experimental process had much more positive effects on Experiment,. The findings indicate that the effect 
of teacher was not found in the experimental groups at the third- grade level, but the teacher effect was observed 
at the fourth- grade level. A t-test was used to analyse the significance of the difference in the gains in ASTS scores 
of the Control, and Control, groups of third- grade students. For the fourth- grade students, the Whitney-Mann U 
test was employed. The results of the t-test and U test are given in Table 13. 


Table 13 
Results of T-Test and U tests of the Gains in ASTS Scores of Third- and Fourth- Grade Students by Group (Control, and Control.) 


Group N X SD df t p 

Control, 30 73 6.46 61 -.02 .988 
Third grade 

Control, 33 16 6.14 

Group N Mean Rank Rank Sum U p 

Control, 42 36.58 1536.50 633.50 .158 
Fourth grade 

Contro! 37 43.88 1623.50 


2 


As can be seen in Table 13, the gains in ASTS scores did not significantly differ for the control groups (Control, 
and Control.) of the third- grade [t,,, = -.02, p >.05]. The gains in ASTS scores of the third- grade control subjects 
are similar. Moreover, the gains in ASTS scores did not significantly differ for the control groups (Control, and 
Control.) of the fourth grade (U = 633.50, p > .05). The gains in ASTS scores of the fourth- grade control subjects 
are similar. There is no teacher effect for the control groups of either third- or fourth- grade. The U test was used to 
analyse the significance of the difference in the EAT retention scores of the Experiment, and Experiment, groups 
of third- grade students. For the fourth- grade students, t-test was employed. The results of the U test and t-test 
are given in Table 14. 
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Table 14 
Results of T-Test and U Test on the EAT Retention Scores of Third- and Fourth- Grade Students by Group (Experiment, and 
Experiment.) 
Group N Mean Rank Rank Sum U p 
Experiment, 23 23.41 538.50 262.50 965 
Third grade 
Experiment, 23 23.59 542.50 
Group N X SD df t p 
Experiment, 20 13.65 5.26 41 -.02 981 
Fourth grade 
Experiment, 23 13.70 7.15 


Table 14 shows that the EAT retention scores did not significantly differ for the experimental groups (Experi- 
ment, and Experiment.) of the third- grade (U = 262.50, p > .05). In other words, the EAT retention scores of the 
third- grade control subjects are similar. The EAT retention scores do not significantly differ for the experimental 
groups (Experiment, and Experiment.) of the fourth grade [tay= —.02, p > .05] either. Therefore, it can be stated that 
the experimental process equally affected the experimental groups. The U test was used to analyse the significance 
of the difference in the EAT retention scores of the Control, and Control, groups of the third- grade students. For 
the fourth- grade students, a t-test was employed. The results of U test and t-test are given in Table 15. 


Table 15 
Results of T-Test and U Test on the EAT Retention Scores of Third- and Fourth-Grade Students by Group (Control, and Control.) 


Group N Mean Rank Rank Sum U p 

Control, 30 30.33 910.00 445.00 489 
Third grade 

Control, 33 33.52 1106.00 

Group N X SD of t p 

Control, 42 9.00 6.03 68.41 -2.25 .028 
Fourth grade 

Control, 37 12.51 7.62 


As can be seen in Table 15, the EAT retention scores do not significantly differ for the control groups (Control, 
and Control.) of the third grade (U = 445.00, p > .05). Students’ being in Control, versus Control, did not have a 
significant effect on their EAT retention scores. The EAT retention scores significantly differed for the control groups 
(Control, and Control.) of the fourth grade [teaaty= —2.25, p <.05] such that the EAT retention scores of the students 
in Control, were much higher than those of the students in Control ,. It can be stated that there was a teacher effect 
for the Control, group. 


Discussion 


Research suggests that drama method has various effects on student achievement, interest and attitudes in 
different courses and subject matters (Abed, 2016; Arieli, 2007; Bailey, 1994; David et al., 2013; Fleming, Merrel ve 
Tymms, 2004, Greenwood, 2001; Kaaland-Wells, 1994; McNaughton, 2004; Vargas, 1995). The drama method is highly 
influenced by the teacher who practices the method. The research model is designed to test whether the teacher 
had an impact on the experiment. It is important to investigate the effect of teachers especially in methods such as 
drama where achieving the intended objectives depends on teacher characteristics and it has been suggested that 
should be analysed (Kaba & Ozdemir, 2012; Ozdemir & Cikla, 2005). For this purpose, with this study the effect of the 
drama method and the different teachers who employed the method on the students were investigated. To analyse 
the teacher effect, a comparison was made on the basis of the data gathered from the EAT, ISE, ASTS and EAT as a 
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retention test. There were two experimental and two control groups in the study. In order to test the teacher effect, the 
experimental groups (Experiment, and Experiment,) were compared, and the control groups (Control, and Control.) 
were compared. The goal here was to test the effect of the teachers on the experimental process and the students. 

The experimental and control groups were compared in terms of the student achievement scores. The findings 
suggest that there was no teacher effect for the third-grade students in improving achievement [t.,,. = —1.17, p > 
.05]. However, for the experimental groups of the fourth grade, there was a positive effect of the drama teacher [U = 
58.50, p < .05], while for the control groups there was a positive effect of the researcher [U = 543.00, p < .05]. However, 
the fact that the same teachers did not make any difference on the experimental and control groups of either grade 
level shows that there was no teacher effect on the experimental process. In other words, the teachers made a dif- 
ference in the experimental groups but not in the control groups, which shows that the teacher is competent in the 
methods but not influential on the experimental process. In order to have a real influence on the method, a teacher 
effect should have been observed in both groups. 

The experimental and control groups were compared in terms of the students’ interest scores in relation to the 
environment. The scores of the experimental groups on the ISE were higher than the control groups’ scores. This 
method increased students interest in the environment. In addition to the effect of the method, teacher influence was 
also observed in the increasing interest of the students. The findings suggest that, for the third-grade students, the 
increase in their interest was significantly related to the drama teacher [U = 136.00, p < .05], while for the fourth grade 
it was related to the researcher [U = 108.50, p < .05]. The effects of the drama teachers who taught the third-grade 
students involved increased interest in the environment. However, for the fourth-grade students, the effect came from 
the researcher, not from the drama teacher, which may be a result of the fact that at this grade level examinations 
and an exam-centred atmosphere were much more dominant. However, this effect of the teachers in the teachers 
was not observed in the control groups. Therefore, the existence of a complete teacher effect cannot be confirmed, 
as there was no significant difference between the scores of Control1 and Control2 groups. 

The data analysis showed that the experimental groups had higher attitude scores than the control groups. It 
can be argued that when the drama method is employed in science courses, students’ attitudes towards science are 
improved. In order to analyse the potential teacher effect, the Experiment1 and Experiment2 groups were compared 
with each other, as were the Control1 and Control2 groups. When the experimental and control groups were compared 
to see whether the attitude scores of the students were affected by the teachers, and no such effect was found for 
the experimental groups of the third grade [t(44) = —.43, p > .05]. However, for the fourth grade, a teacher effect was 
observed for Experiment1, which was taught by the researcher [t(41) = 5.10, p < .05]. There was no such effect for the 
control groups of the third grade [t(61) = —.02, p >.05] or the fourth grade [U = 633.50, p > .05]. It was also found that 
there was no teacher effect on the significant differences observed in the science attitude scores of the third-grade 
students. However, the same effect was not observed for the control groups. Therefore, it can be stated that there was 
no genuine teacher effect on the experimental process; rather, the teacher effect influenced the research through 
the method employed. This result, which was also observed in student achievement, shows that both teachers had 
the same effect on the experiment. The drama method was influential in increasing student attitudes, but a teacher 
effect was not observed. However, for the fourth-grade students, the science attitude scores of the class taught by 
the researcher were higher than for the other experimental group, which may indicate a teacher effect. The students 
knew the researcher as a teacher at the school. Therefore, they might have considered the activities much more seri- 
ously and thus shown much more interest and participation in the activities. 

The experimental and control groups were compared among themselves to determine whether there was 
a teacher effect on the retention of environmental information. The results showed that both the third-grade ex- 
perimental subjects [U = 262.50, p > .05] and the fourth-grade experimental subjects [t.4, = —-02, p > .05] did not 
significantly differ. There is a meaningful difference between the control groups of the fourth grade in favour of the 
Control, group, in which the drama teacher taught the classes [toa = —2.25, p <.05]. Therefore, it can be said that a 
teacher effect was found in the delivery of the traditional lesson plans at the fourth-grade level. However, it is not 
possible to talk about the effect of the teacher on the experimental environment, since there was no such difference 
in the experimental group where the drama teacher taught the classes. It can be said that the teacher factor made a 
difference in the students’ retention scores. 

In terms of the teacher effect, it can be stated that there is no teacher effect by method in the third and fourth 
grade. The researcher was influential in employing the drama method to increase the science achievement scores 
and science attitude scores of fourth- grade students. The drama teacher seems to play a significant role in increas- 
ing the retention scores of fourth- grade students over the traditional method of application. For the third- grade 
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students, the creative drama method and creative drama teacher were effective in increasing their scores on interest 
in the environment. 

The success of the drama method includes the teacher's approach to the method, teacher willingness and com- 
petency to employing the method. If teachers have the adequate information and skills for drama method, the effects 
of the drama activities will be so great. Negative attitudes and beliefs about the method, inadequacy of planning and 
using the method may adversely affect drama applications. McNaughton (2004) concluded that the students learned 
much new knowledge through the creative drama method and had fun at the same time. McNaughton (2004) found 
that the students defined creative drama as an educational activity that teaches them while entertaining them. David 
et al. (2013) concluded that the creative drama method facilitates student participation and makes them interested in 
the topic. Levey (2005) states that as a teacher who used it throughout the curriculum, he found the creative drama 
method particularly useful in environmental education. He further states the advantages of the method in learning 
environments as follows: 


“The use of creative drama in environmental education can make a huge difference in learning environmental issues. 
When students dramatize a topic, they begin to discover it from different perspectives and to develop relationships with 
it in a creative way. In doing this, students not only provide profound learning and retention in knowledge, but also gain 
a deep insight into themselves, other people, and even other creatures. Drama empowers students to expand their world 
experiences by empathizing with their surroundings to protect the environment (p.35).’ 


Conclusions and Implications 


In short, it can be said that there are many factors affecting the success of the drama method such as an exam- 
oriented education system, student-teacher relationships, the socio-economic environment, and family demands. In 
this research, the effect of the teacher on the success of the method was studied. In general, the method had a positive 
effect on the students’ attitudes towards science, their attitudes towards the science course, and their knowledge. 
Whether one of the teachers had an impact on this positive effect was also examined. It was found that the teacher 
was more effective in the methods but had no effect on the experimental procedure as a whole. More specifically, a 
teacher effect on the method was observed for the third-grade students, but a researcher effect was found for the 
fourth grade. However, the effects of the method cannot be neglected. 

The achievement of the third-grade students participating in the study who were taught science education 
through the creative drama method for the first time was improved, and this positive effect was not a result of the 
teacher effect. The creative drama method was effective for students’ achievement, interest, and attitudes in both 
experimental groups (Experiment, and Experiment,). Given that the third-grade students were being taught science 
education for the first time, they had no expectations, which had positive effects on their results. 

The findings of the study indicate that the achievements, interest towards the environment, and attitudes to- 
wards the science course of the third-grade students who received the course through the creative drama method 
were higher than for the control group students who were taught using the traditional method. This situation shows 
us that the drama method, which is one of the cornerstones of constructivist theory, should be used at all levels of 
education, starting from pre-school education. 


Recommendations for Practitioners and Future Researchers 


Discussions of environmental issues cover some environmental problems that seem to make students pessimistic. 
Therefore, environmental education should be taught to younger students using such methods as creative drama, 
which can positively affect the attitudes of students towards environment. Environmental issues should be taught 
using those methods that will enable the student to be active in daily life starting from the students’ immediate sur- 
roundings and cultivating students’ love of plants and animals. As environmental issues can be covered in almost any 
course, it is suggested that these issues be given in relation to all topics. Given that the success of the drama method 
depends on the teacher's success, teachers should participate in in-service training activities and other workshops 
organized to improve their knowledge and skills regarding the drama method. The drama method should be frequently 
included in the activities in textbooks and the recommended activities in teachers’ guidebooks. Thus, teachers can 
become more interested in the subject. Increasing the number of hours devoted to teaching environmental issues will 
allow teachers to devote time to the methods that require more time to apply, such as creative drama. If the drama 
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method is to be implemented for the first time in a group, the first few sessions should be planned to introduce the 
method. Having a hall or class specifically devoted to creative drama in schools will make the implementation easier 
and more practical. Classes with fewer students should be chosen for the implementation of the method, as creative 
drama practices will be difficult in crowded classes. 

The results of the implementation of environmental education among younger students can be investigated and 
compared with the results of previous studies of samples of students from different age groups. The teaching methods 
and environmental attitudes of the teachers in primary schools regarding environmental issues can be investigated. 


Limitations of the Study 


This research has some limitations: This study, with 2015-2016 academic year; with a public primary school where 
the application will be carried out, with the 3rd and 4th year students who took creative drama method; with the 
environmental objectives and creative drama method in the 3rd and 4th grade curriculum; with pre-test, post-test, 
pre-interest, post-interest, pre-attitude, post-attitude and retention test scores was limited. 
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A Sample Creative Drama Course Outline 


Plan Label: Blue Lake 


Method: Creative drama 


Course: Science Place: Multi-purpose hall 
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Topic/Concepts: School and environment, environmental cleaning, 


environmental pollution, environmental protection and beautification 


Techniques: Role playing, improvisation, group pantomime, dull 
image. 


Students: 3rd. and 4th grade students 


3rd Grade/Classes: 3E-3C 

Unit: Journey to the World of Living Beings/Living Beings and Life 
Topic: Me and My Environment 

Date:28/04/2016 Duration:40+40+40 mins 


4th Grade/Classes: 4B-4E 

Unit: Microscopic Creatures and Our Environment/Living Beings and 
Life 

Topic: Human and Environmental Relations 

Date:29/04/2016 Duration:40+40+40 mins 
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Learning Objectives 


3rd Grade: 1. Students recognize the environment in which they live and 4th Grade: 1. Students keep their surroundings clean to avoid 

take an active role in the cleaning. 2. Students understand the importance —_ environmental pollution. 2. Students discusses how environmental 

of the natural environment for living things and take measures to protect. pollution can be prevented. 3. Students design a project to protect and 
beautify the environment. 


Creative drama objectives: 1. Students respect the performance of their friends in the improvisation. 2. They could work with others. 3. They 
can work in a small group. 4. Students can evaluate others’ work. 5. Students can take part in whole class improvisation. 6. Students can communi- 
cate with their peers. 7. Students can design improvisation according to a given topic. 


Materials: 1. Pictures showing clean days of the lake. Cardboard papers for each group 2. Written sentences on small slips of paper on air and wa- 
ter pollution. Pictures of air and water pollution. 3. CD player; for the 4th activity, Jazz Ensemble, The Taking of Pelham. 4. For the group pantomime, 
Taraf De Haidouks & Kocani Orkestar, -A la turk is used. 


Preparation: The impact of a clean or dirty immediate environment on human health and development is emphasized. It is also emphasized that 
a clean and green environment makes people happy. The teacher sounds out the opinions of the students on how dirty air and dirty water adversely 
affect health. 


Preparation/Warm-up 


1ST ACTIVITY: Game of Air-Soil-Water: Children stand in a circle. When the teacher says “air,” they act as if they are flying; when s/he 
says “water,” they act as if they are swimming; and when s/he says “soil,” they act as if they are digging something. 


2ND ACTIVITY: Types of pollution: Children stand in a circle. Each student is given a label of a type of pollution. One student is chosen to 
be “it” and stands in the middle of the circle while trying to find a place in the circle during the game. Students are reminded not to forget the names 
they are given. When the game starts, one of the pollution types is called out. Those whose labels are called try to change their places in the circle. If 
“it” wants all of the children to change their places, “it” says “pollution basket” and tries to take the place of one of the children in the circle. If he can 
take someone's place he is no longer “it,” but remains “it” otherwise. The game continues in this fashion. 


3RD ACTIVITY: Mid-Evaluation: They talk about the answers to the following questions: How is air polluted? How is water polluted? What 
pollutes the air and water? How is the soil polluted? What should we do to avoid contamination of our environment? 


4TH ACTIVITY: Find the Sentence—Find Your Pair: The students stand in a circle. Two copies of the sentences are prepared and 
written on small pieces of papers. These sentences are about either the reasons for air pollution or the ways to avoid air pollution. Then each of 
these pieces of paper is attached to one student. The students are asked to find the other student with the same sentence as theirs. The pairs stand 
where they are. Once the pairs of children are formed everybody reads their sentences one by one. The papers of the students having the second 
sentences are turned down and worn by the students. This time some of the students write the reasons for air pollution and the others the ways to 
avoid air pollution. Problems are matched with the right solutions. 


5TH ACTIVITY: Outdoor Activity: During the previous drama activity, there was a survey questionnaire, “Let's Decorate the Newspaper 
Building with Flowers.” There you were asked to state your favourite outdoor activities. Do you remember? What were these activities? Which activi- 
ties were voted most highly in the survey questionnaire? 


Where do you do during these outdoor activities? Are there any places where you can have a picnic and ride a bike near your home? A picture of the 
“Blue Lake” is shown to the students, and the teacher may tell them that they all know the Blue Lake. Then the following question is asked: What do 
people usually do at the Blue Bake? Students list the activities that can be performed at the Blue Lake. Playing ball, skipping rope, fishing, picnic with 
your family, etc. What else can be done at the Blue Lake? Let's think about it. The activities are then listed on a large cardboard paper. 


Role Play/improvisations 


6TH ACTIVITY: Still Image—I am at the Blue lake: Students are told to draw an activity that they would like to perform at the Blue Lake. 
They animate this activity through still image techniques. Each student animates the activity he / she wants to do. Some of the students are asked to 
tell what they want to do. Some students want to fly a kite, some to ride a boat. 


* Some of them drew fishing, some having a picnic, some cycling, some traveling, some playing ball, some cooking kebabs. Some of them were visit- 
ing the place with their grandfather. 


7TH ACTIVITY: Group Pantomime: Students are told to form a group by observing each other. Then the students in pairs or individually 
begin to animate the still images. 


8TH ACTIVITY: Estimation: An estimation task is performed with the rest of the class. The teacher determines the four different activities that 
are performed by the students and points out four students who perform these activities well; the others are then asked to sit down. The four students 
are told to stay on the stage and continue their movements. The class is asked to think about which outdoor activities are being performed by these 
students. For instance, the groups attempted to portray “Look, there is an old man fishing here; who wants to go there?” In this way the other students 
are also taken to the stage. 


+ Most of the groups had more spontaneous reunions. For example, the cyclists were tired and stood next to the fishermen, and the kids playing ball 
were the children of the picnickers, etc.... The groups were formed in this way and the class was divided into groups of four. 


9TH ACTIVITY: A Day at the Blue Lake: How do these four groups spend a day at the clean and orderly Blue Lake? The students are 
asked to think about it and animate part of it. Then, the students who had roles as picnickers, cyclists, kite-flyers, and fishermen animated how they 
spent a day at the Blue Lake. 


While making the improvisation, relations began to develop among the still images in the sixth activity. All groups performed the improvisation suc- 
cessfully. The teacher wandered among the groups to see whether the students needed help. It was observed that the students were very harmoni- 
ous and experienced because they had the Blue Lake experience during the improvisation. 609 
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Evaluation 
10TH ACTIVITY: Slogan Writing: Students produce slogans describing the effects of fresh air and abundant oxygen on human life. The 
slogans are turned into banners, and then the slogans with the most votes will be displayed in the school. Some of these slogans are; 
“Clean your environment for protecting our planet” 
“Clean air means healthy people” 
“Cleanliness is happiness” 
Assignment: Children paint pictures of the potential activities that may be performed at the Blue Lake. 


Attachment 2 
Observation Form 


Dear Researcher/Practitioner 

You involved in the courses of environmental education to the third and fourth grade students. Course outlines 
used for the experiment groups for which a creative drama method was conducted are given in attachment. Please 
carefully examine these outlines. Please answer the items below considering that a creative drama approach was 
used for the experiment groups, and it was not used for the control groups. Your answers should be expressed 


through one of four options, namely “good", “average”, “not good” and “N/A” Puta sign on () which is at the right side 
to indicate your answer, like (X). If you are asked to give the number of the activity, please give it on the proper box. 


No: Observation Questions Good | Average} Bad NIA 


Did the group make a circle? 


1 

2 Was there a mental preparation? 
3 Was there a physical preparation? 
4 


Was the duration of the preparatory work sufficient? For a course duration of 40+40+40, 
were there preparations for 30 minutes? 


Were the warm-up activities made related to the objectives? 


Were improvisations performed? 
Were the activities made related to one another? 


Was the preparation made before the improvisation? 


O;]aol;l NI ony; an 


Were the improvisation activities carried out in each lesson related to the objective of the 
related lesson? 


x 


Were dramatic situations in the animations given clear? 
Was a dramatic situation used for each objective? 


x 


x 


Were the roles in improvisation clear? 


x 


Were the purpose, status of and motivation for roles clear? 
Were the drama techniques used in the improvisation stage? 


x 


Was the duration of the improvisation proper? 
Were the components of the dramatic fiction complied with? 


x 


x 


Were time and space clearly stated for the improvisations? 


x 


Did the teacher play any role in improvisations that did not create conflict situations? 
Were the instructions given by the teacher clear and explicit? 


x 


x 
O}/OI NID! an] ei[wowl]rm]-—-|o 


NO 
Oo 


Was the intra-group communication sufficient? 


Was the duration of the improvisation work sufficient? For a course duration of 40+40+40, 
were there improvisations for 60 minutes? 


DO 
= 


DO 
ie) 


Was the role-playing technique used? 
Did students stand up during the course? 


DO 
wo 


DO 
BS 


Did the students sit on the ground from time to time during the course? 


DO 
or 


Was the space used in a seated order? 
Was music used during the course? 


nO 
fo>) 
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No: Observation Questions Good | Average} Bad NIA 

27 Did students occasionally stand up during the course? 

28 Was the course delivered at the multi-purpose hall? 

29 Was the school yard used during the course? 

30 Were drama course outlines used? 

31 How do you rate the implementation of the daily plan by the teacher? Please choose the 
statement “the course was delivered exactly as stated in the daily plan’ if you have posi- 
tive views and choose the statement “the course was not delivered as stated in the daily 
plan” if you have negative views. 

32 Were the body of the students used as a material? 

33 Were there mid-term evaluations? 

34 Was there any evaluation at the end of the course? 

35 Did they play games? 

36 Did the teacher use direct instruction method? 

37 Did the students sit in a row? 

38 Were the students active during the course? 

39 Did the students interact with one another? 

40 Did the students freely walk around the space? 

41 Did the students use all parts of the space? 

42 Did the students move with music? 

43 Was any classroom used for the course? 

44 Were the products of the children developed in the course exhibited? 

45 Was the evaluation associated with achievements? 

46 Was the teacher used a textbook or printed material? 

47 Was any self-evaluation used? 

48 Was any peer evaluation used? 

49 Was any group evaluation used? 

50 Did the students play games from time to time during the course? 

51 Was the teacher warm and friendly? 

52 Was the teacher effective in involving students in activities? 


The points you want to add: 


Positive points: 


Negative points: 


Other: 
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Abstract. Teachers use different pedagogies 
to improve learners’ performance. The study 
explored the effect of Computer Simulations 

(CS) on Grade 11 learners’ performance when 
taught Plants Biodiversity. A Solomon Four- 
Group design was used to cater for internal 
and external validity. Sixty-six learners were 
assigned to two Control Groups (CG) taught 
using CS and 66 learners to two Experimental 
Groups (EG) taught using Talk and Chalk 
Method (TCM). The pre-test was adminis- 
tered to EG1 and CG], while post-tests were 
administered to all four groups. Focus Group 

Discussion Interviews (FGDI) were conducted 

with 12 learners: six from EG and six from 

CG. Quantitative data were analyzed using 

a T-test, Analysis of Variance (ANOVA), while 
qualitative data were analyzed using themat- 
ic analysis. The results show that EG outper- 
formed CG (T-test; ANOVA; p < .05). Boys’ and 
girls’ performance in EG did not differ signifi- 
cantly, suggesting that CS favour both gender 
to perform well. CS positively influenced EG 
learners’ attitudes towards Biodiversity topic, 
but not CG. Thus, CS is an effective tool for 
enhancing learners’ performance. 

Keywords: computer simulations, Solomon 

Four-Group Design, learners’ performance, 
Talk and Chalk Method (TCM) 


Israel Kibirige, 
Kgashane Bethuel Bodirwa 


University of Limpopo, South Africa 


612 


@® 63) This is an open access article under the 
«c) Creative Commons Attribution 4.0 
BY _NC International License 


THE EFFECT OF USING 
COMPUTER SIMULATIONS 
ON GRADE 11 LEARNERS’ 
PERFORMANCE IN PLANTS 
BIODIVERSITY IN SOUTH 
AFRICA 


Israel Kibirige, 
Kgashane Bethuel Bodirwa 


Introduction 


Improving learners’ performance is a goal in teaching science subjects. 
Science teachers have diverse knowledge to teach different topics (Shulman, 
1986). The teachers’ knowledge to teach different topics is Pedagogical Con- 
tent Knowledge (PCK) (Deidre, 2015; Shulman, 1986). PCK is distinct knowl- 
edge exclusive to teachers, distinguishing them from the subject content 
specialists (Shulman, 1986). Although researchers in teacher education have 
ignored PCK, Hill, Ball and Schilling (2008) report that the PCK teachers use 
can improve teaching and learning. Educational research shows that teach- 
ers’ creativity in teaching affects students’ learning (Kleickmann et al., 2013). 
Teachers direct special attention toward science topics: Content Knowledge 
(CK) and Pedagogical Content Knowledge (PCK) (Sickel & Friedrichsen, 2017). 
Both types of knowledge affect teachers’ instructional practices and student 
learning (Baumert et al., 2010; Hill et al., 2008). Shulman (1986) states that 
PCK is a cognitive construct representing an idea embedded in the teachers’ 
beliefs. Itis more than teaching content knowledge to learners since learning 
is not absorbing information for reproduction in the exams. 

PCK is the fusion of content with methods of teaching (1986, 1987). 
This knowledge is not the same for all teachers, but it is explicit knowledge 
within a teacher's idiosyncratic practice. Thus, teachers can choose different 
teaching tools to assist them to enhance learning. According to Lowther et 
al. (2012) projectors, televisions, and computer laboratories are forms of tech- 
nology to enhance learning. Using computers in teaching science is popular 
in the developed world and is slowly picking momentum in the developing 
world. Technology stimulates learning and more so, learners can manipulate 
objects using different devices (Kirkley & Kirkley, 2005). Also, teachers employ 
technology in numerous ways to address learners’ needs (Koh et al., 2018). 
One approach is to use computers, which motivate learners. Also, teachers 
use Computer Simulations (CS) for teaching and for updating their content 
knowledge. 

CS engage learners in understanding artificial as well as natural systems 
(Ramat & Preux, 2003). During simulations, students are the test subjects. 
CS contribute to the ‘learners-play-to-learn’ process where learners create 
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knowledge through playing (Mavhunga & Kibirige, 2018), increases learners’ interest (Yildirim & Sensoy, 2018) and 
enhances learners’ knowledge retention (Popil & Dillard-Thompson, 2015). CS improve learning outcomes in the 
science classroom (Akpan, 2001) and are essential in research and investigations (Samsonau, 2018). Hannel and 
Cuevas (2018) reviewed teachers’ use of CS and found that they assist learners in time management because less 
time was devoted to setting up apparatus. Besides, in EG learners changed variables to allow stating and testing 
hypotheses. In this context, a learner becomes active and improves in formulating questions, hypothesizing, col- 
lecting data, and revising theories. Interactive CS provide learners with a sense of ownership and thus increase 
content understanding as well as content retention (Ramos et al., 2016). Simulations allow learners to reproduce 
and envisage actual world processes that would take a long time or perilous processes in a high school laboratory 
context (Akpan, 2001). 

Limniou et al. (2007) contend that replacing some laboratory lessons with cooperative pre-laboratories simula- 
tion increases learners’ knowledge. Similarly, Dalgarno et al. (2009) used a 3-dimensional Virtual Laboratory (VL) and 
a Real Laboratory (RL) to acquaint learners with the long- term structure of laboratory apparatus. They concluded 
VL was effective for familiarisation with laboratory background. However, several studies opine simulations should 
not substitute physical laboratories (Limniou et al., 2009; Sypsas & Kalles, 2018). CS motivate learners to interact 
with real-life issues. Using CS, the teachers can regulate parameters to achieve the best learning outcomes (Hertel 
& Millis, 2002). Learners engage with simulations to alter parameters, and practice solving specific tasks. Thus, CS 
can be incorporated into learning to discover hypothetical situations. Learners can compare the information from 
CS with textbooks and notes from lectures to solve problems in a realistic mode to minimise learning difficulties 
(Samsonau, 2018). 


Research Problem 


Teachers use Pedagogical Content Knowledge (PCK) for specific topics and different contexts. This knowledge 
defines them as teachers. Sanders, Borko, and Lockard (1993) assert that experienced teachers have affluent PCK, 
while Chan and Yung (2018) indicate teachers’ previous experiences may promote or hinder their new PCK devel- 
opment. Studies confirm teachers seldom use technology in addition to their PCK as Technological Pedagogical 
Content Knowledge (TPACK) to improve learning (Mavhunga et al., 2016). Teachers spend a long time talking in 
class and ask low order questions (Carlsen, 1993). Some teachers cannot identify learners’ misconceptions (Hash- 
weh, 1987) and consequently choose incongruous strategies to teach content. One strategy to improve learners’ 
performance is to use CS. The effect of using CS on learners’ performance has not been well studied, particularly in 
developing countries, where learners-to-computer ratio varies. For example, in Zambia, the learners to computer 
ratios were as high as 143:1 (Chaamwe, 2017; Sossa et al., 2015) and 17:1 in South Africa (Kibirige & Hodi, 2019). 
It should be noted that in South Africa as a developing country, learners are challenged in two aspects: 1) lack of 
access to the use a computer, learners cannot afford to own one (Habibi et al., 2018), and 2) learners have limited 
time to practice on the computer because they typically access the computer during school hours (Tarman & Chi- 
gisheva, 2017). Some content cannot be taught using hands-on activities in the classroom because of its abstract 
nature. In such instances, CS can be viewed in the classroom to minimize misconceptions on a specific topic. After 
all, misconceptions are context and topic-specific, making this study unique because, to the best of our knowledge, 
no studies used CS to teach a Plant Biodiversity topic in a rural school context in South Africa. This is a knowledge 
gap this study aimed to fill. Thus, the study investigated the effect of CS on learners’ performance in Plant Diversity 
compared to the learners taught using (TCM). 


Research Focus 


The research focuses on the effect of CS, which engage learners in active participation more than the Talk 
and Chalk Method. The research contributes to literature from developing countries on learners’ experiences of 
using CS to improve their understanding of science. CS assist learners to improve their attitudes and interests, and 
cognitive achievements in science (Tlystiz, 2010; Yildirim & Sensoy, 2018). Considering the vast majority of teach- 
ers using TCM in schools, which results in learners’ poor conceptualisation and poor attainment in sciences, CS 
would change the paradigm and actively engage learners, especially during the COVID-19 era where face-to-face 
interaction is minimal. Thus, the findings from this study can benefit Life Science teachers, learners, curriculum, 
and material designers in chemistry to incorporate CS in the teaching of Life Sciences. 
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Research Aim and Research Questions 


Due to the scarcity of computers in schools in South Africa, many learners are assigned to one computer 
to access during their free time. The effect of using CS to improve learners’ performance in Life Sciences in de- 
veloping countries with limited computers is not well studied. Thus, the purpose of the study was to explore 
the effect of CS on Grade eleven learners’ performance in a Plant Biodiversity topic. The study was to answer 
one question: What is the effect in learners’ performance of teaching Grade 11 learners Plant Biodiversity using 
CS? The study addressed three hypotheses: 1) Pre-test learners’ scores vary in EG and CG; 2) EG learners taught 
using CS perform better than CG learners taught using TCM; 3) There is no statistically significant difference in 
learners’ performance between boys and girls in EG. 


Research Methodology 
General Background 


The researchers used Solomon Four-Group Design (Cambell & Stanly, 1963) because of its robustness. It deals 
with internal validity, where one considers alternative causes, and external validity, where the results can apply 
to the entire population (Kirikkaya & Basgtil, 2019). The design assists in 1) identifying the cause and effect of CS 
intervention. 2) The researcher can determine whether the administration of a pretest affected the dependent 
variable (Abaniel., 2021). Thus, this study differs from earlier quasi-experimental designs (Tlala et al., 2014), which 
did not cater for these two threats and hence results were not conclusive of the cause and the effect regarding 
the use of Predict-Observe-Explain (POE). The current design addressed the effect of Computer Simulations us- 
ing a Biodiversity topic in rural South African schools during 2018. The research design is represented in Table 1. 


Table 1 
A Solomon Four-Group Design (EG, CG, pretested and EG, CG, EG, CG, post-tested) 


Randomized Group Pre-tested Intervention Post-tested 
EG, Yes Yes Yes 
CG, Yes No Yes 
EG, - Yes Yes 
CG - No Yes 


The target population was 200 Grade 11 Life science learners aged between 15 and 17 years with an aver- 
age age of 16 years were purposively selected (Creswell, 2013) from four schools based on the availability of 
computers. The Raosoft Software sample calculator provided a sample of 132 learners, with a 6.95% margin of 
error at a Confidence Interval (Cl) of 95%. This implies the sample was a representation of the population and 
therefore suitable for the study. 132 Grade 11 learners were randomly assigned to four groups: two groups with 
33 per group as EG and two groups with 33 per group as CG. The researchers considered 33 learners per group 
adequate (Mugenda & Mugenda, 1999). 


Instruments and Procedures 


Pre- and post-tests and Focus Group Discussion Interviews (FGDI) were used to collect data. 

The tests were checked for face validity by two lecturers and piloted to 20 learners from one school with 
a similar environment, which was not part of the study. For reliability, a Cronbach Alpha coefficient of .85 was 
obtained, which is acceptable because it is above .7 cut-off line (Muijs & Reynolds, 2011). The interview schedule 
questions were checked for face validity by two lecturers, and their recommendations were addressed before 
data collection. 
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EG, and GG, were pre-tested, EG, and CG, were not pre-tested (Table 1), and the post-test was administered 
to all the four groups EG,, EG,, CG, and CG, Pre-post-test learners’ scores were the independent variables. EG, 
learners were taught using CS to influence outcomes to assist researchers in assessing the impact of the treat- 
ment. Learners were taught during the usual school periods to avoid school curriculum disruption. The researcher 
accessed simulations from Physics Education Technology (PhET) online. Although the name appears as physics, 
the simulations include other subjects like Biology, Chemistry and Earth sciences. The topic was the main group- 
ings of living organisms and their diagnostic features such as bryophytes, pteridophytes, gymnosperms and 
angiosperms. Although the regular class teachers were present during the teaching, they did not participate in 
the teaching. The researchers led the class in discussions and in answering questions regarding the simulations. 

Four steps were followed during the lessons: 1) general introduction of the content - learners wrote their 
predictions on what Kingdom or Division the organism belong; 2) screening of Computer Simulations - learn- 
ers observed; 3) reflections on the simulations - learners wrote explanations and discussed within the class; 4) 
wrapping up the lesson- the teacher clarified a few misconceptions and highlighted main ideas of the lesson. 
After an introduction, learners wrote their predictions following teachers’ guides. During simulations they ob- 
served and identified different plants and classified them into their Divisions. For example, Bryophytes, learn- 
ers predicted if the plant according to the structure was vascular or non-vascular; Pteridophytes - Spores but 
no seeds, Spermatophytes- Seed Plants, and Gymnosperms - naked seeds. The above content was in line with 
the Grade 11 curriculum. The study lasted for five weeks, where one week was used for acclimatization. Due to 
the shortage of computers in the schools, the learners-to-computer ratio was limited to four or five learners 
per computer and learners accessed computers at their opportune time after classes. Focus Group Discussion 
Interviews (FGDI) with 12 individuals (6 per group) were employed to collect the views on learners’ attitudes 
before and after teaching. Each interview lasted approximately 30 minutes and was audio recorded. Four ses- 
sions were held, and the interviews ended when there was no extra information gained from the discussions. 


Data Analysis 


Quantitative data were analysed using means, standard deviations, T-test, and Analysis of Variance (ANOVA). 
Levene Test for Equality of Variances was applied to the pre-test scores to determine if the two groups (EG and 
CG) were equal. T-test was used on pre-test to detect equality of the two groups, and pre- and post-tests mean 
differences between EG and GG, and the gender differences in performances in EG. A 2 x 2 ANOVA test on four 
post-test scores (EG,,CG,, EG, and CG.) was used to determine differences between the CG and EG. Finally, qualita- 
tive data were analyzed thematically using six steps of Braun and Clake (2006): reading line by line through the 
transcripts to be conversant with the texts, creating tentative codes, looking for themes, revising the themes, 
defining themes, and finally writing the themes. 


Ethical Issues 


The Education Department granted permission to carry out the study. All learners consented to participate, 
and their parents signed consent forms. During data collection, anonymity and confidentiality of participants 
were ensured throughout the entire study. To minimize learners’ discrimination from a worthwhile teaching 
method, the other two classes taught without CS were given extra classes after the five weeks of intervention 
to experience the CS approach 


Research Results 


The results show that EG taught using CS performed better than CG, which was taught without CS. Lev- 
ene Test for Equality of Variances F =1.64, p> .05) showed no significant differences between the pre-test scores 
of EG and GG. The statistics related to equal variances were assumed, and hence the use of the t-test. 

The results of pre-tested EG, and CG, using an independent T-test are presented in Figure 1 a-b. 
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a) T-test results for pre-test for EG, and CG, groups before intervention. 
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6) T-test results for post-test for both EG, and CG, groups after intervention. 
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In Figure 1 a) pre-test results for CG (M = 12.94, SD = 4.90) and EG (M = 14.07, SD = 3.78) showed no significant 
differences T-test, t(64) = 1.14, p > .05). Thus, the learners had a similar understanding of science concepts before 
the intervention. Therefore, we reject Hypothesis one, which states that learners’ scores vary in the pre-test. 

After teaching for four weeks, the post-test results of EG, and CG, performances were compared using a T- 
test. Figure 1 b) shows results of the post-test scores for both EG, (M = 38.53, SD = 11.19) and CG, (M = 22.32, SD 
= 6.87) differed significantly after intervention T-tests, t(64) = 7.19; p < .05). Also, Cohen d results for EG, and CG, 
show a huge effect size of 2.02 for the EG, group compared to 1.6 for the CG. . Effect size between the EG, and CG, 
A2 suggesting EG, performed better than the CG,. Therefore, Hypothesis two, stating learners’ performance does 
not vary in EG, and the GG, after intervention is rejected. 
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Figure 2 


T-test Results for Post-test for Males and Females in EG after Intervention 
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Figure 2 shows mean scores were (M = 39.63, SD = 11.52) for males and (M = 38.67, SD = 12.08) for females. No 
statistically significant differences were found among the two groups using a T-test, t (64) = (.39); p > .05. Therefore, 
hypothesis three stating scores vary between boys and girls in EG after intervention is rejected. 

ANOVA results of the treatment are represented below in Table 3. 


Table 3 
ANOVA Representing the Main Effect Test between EG and CG 


Solomon Four Group Type Ill SS df MSs F p 
Treatment versus No treatment 7592.86 3 2559.55 25.80 01 
Pre-tested versus Not Pre-test 86.11 1 124581.97 1255.71 01 
Treatment versus pretest 7.21 1 86.11 87 35 
Error 7686.18 1 7592.86 76.53 01 
Total 144378.00 127 99.21 07 19 
Corrected Total 20278.58 131 


In Table 3, the significance level (p-value) corresponding to treatment*pretest is .35, which is more than the .05, 
suggesting there is no interaction effect. The p-values corresponding to the Pre-tested versus Not Pre-tested and 
Treatment versus Not treatment are less than p = .05. Thus, there is a main effect between CG, and EG,, and between 
pretested and not Pre-tested groups. Also, between groups variance is not greater than within groups variance 
because the F = .87 value is small. 

In the qualitative results, three themes emerged from FGDI: 1) interest in learning, 2) acquisition of knowledge, 
and 3) finishing tasks in time. Each theme is presented below using learners’ narratives. 


Theme 1: Interest in learning 


The main excerpts describing learners’ attitudes and interests consisted of words such as: boring, not fun, 
easily forgettable, and difficult. Learners’ verbatim quotes are presented here below: 
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FGDI 1:“Learning Life Sciences in boring and it is not an interesting subject”. CG 
FGDI 3: “The subject is boring. We study for a pass only, and it is not even fun.” CG 
FGDI 4: “This time | found Life Sciences interesting since we learnt Plant Biodiversity using simulations.” EG 


Theme 2: Acquisition of knowledge 


From the discussions, learners indicated learning using CS assisted them to comprehend Plant Biodiversity 
concepts better than before. Learners could replay simulations at their own time to master the content. 


FGDI 1:“I discovered Life Sciences interesting, | remember everything regarding the content in the class using CS.’EG. 
However, learners had challenges in remembering the taught content. 


FGDI 2:“We forget the taught content, and are unable to share the content we learnt” CG 


Theme 3: Finishing tasks on time 


When learners were asked to comment on the strategy used in teaching, the following direct quotes from 
FGDI are presented here below: 


FGDI 3:“l enjoyed Life Science lessons because | spend more time studying Life Sciences” EG 
FGDI 4: “We do not cover all expected topics as specified in the pacesetter and it is hard for us preparing to write 
exams.” CG 


Discussion 


This study explored the effect of CS on Grade 11 learners’ performance in Plant Biodiversity. The results show 
learners in the EG, achieved significantly higher scores than learners in the CG,. The study showed statistically sig- 
nificant differences in the performance between learners taught using CS compared to those taught using TCM. This 
suggests hypothesis one stating learners in the EG taught using CS perform better than those in the CG taught using 
TCM is accepted. Our findings agree with Ragasa (2008), who showed computer-assisted teaching and learning is 
more effective than TCM. When teaching using CS, learners are accountable for their learning. Learning occurs through 
different levels where learners assess themselves and concentrate on meaningful learning. This observation agrees 
with the constructionists’ theory, where learners use their environment to enhance their existing knowledge to learn 
effectively (Vygotsky, 1978). In fact, Gonczi et al. (2017) stated that CS teaching is twice as effective compared to TCM. 
It is no wonder learners using CS performed better than learners taught using TCM. 

The results show EG learners taught using CS scored higher than CG learners taught using TCM (Figure 1 a). EG 
learners worked together to enquire, improve their cognition, and interact with their peers to study Plants Biodiversity. 
This observation agrees with the social constructivism theoretical framework where learners interact with objects and 
their peers to learn. Similarly, the technology used blended well with the pedagogy and content knowledge in teach- 
ing EG using CS. Technology also offered a conducive learning environment that facilitated interest, collaboration, and 
social construction of knowledge. Learners in EG scheduled their time to access CS on the school premises, so learners 
self-regulate their learning. It is a unique finding because no studies on high school biodiversity have been reported 
in the South African context. 

There were no statistically significant differences in mean scores of male and female learners (Figure 2). It can be 
concluded that both boys and girls achieved similar conceptual understanding when taught using CS. Thus, hypothesis 
three which states that boys’ and girls’ performances in EG vary is rejected. The findings concur with the studies by 
Okwuduba et al. (2018) and Mihindo et al. (2017), who investigated the impact of computer simulations on gender 
and found no significant differences among male and female secondary school learners’ performance in stoichiometry. 
They attributed the improvement to computers used in the activities. Fraser and Walberg (2005) contend that using 
technologies in teaching improves learners’ performance. 
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Learners in CG did not perform as well as EG because they might have lacked the enthusiasm to learn Life Sciences 
as it was rated as boring. Conversely, learners from EG found CS very interesting. These observations agree with Yildirim 
and Sensoy (2018), who indicated dynamic interactive visual representations are necessary to enhance academic per- 
formances and learners’ interests. They further alluded that attitude is one of the critical factors determining learners’ 
performance in science. EG group learners regarded the lessons as interesting and remembered what they had learnt 
using CS. This account was clear when learners shared information among themselves after the lesson presentations. 
The improvement in conceptual understanding corroborates with Popil and Dillard-Thompson (2015), who found 
simulations enhanced learners’ knowledge acquisition. On the one hand, EG learners indicated that using CS helped 
them cover a wide scope quickly, giving more time for revision. On the other hand, learners from CG were frustrated 
because they did not complete the curriculum. Popil and Dillard-Thompson (2015) highlight using TCM limits learners’ 
academic success by depending on teachers. If the teacher is not at school, learners study nothing during that time. 
Besides, teachers may hamper the academic performances of gifted learners. It is unlikely that teachers would revise all 
the content with learners. 

Comments from EG show learners were excited about the use of CS to cover the scope with confidence. The 
excitement may have increased their desire to do more and achieve more. These results support the TPACK theoretical 
framework where instruments’ interaction with social beings yields high performance (Koehler et al., 2014). Few studies 
in education used the Solomon Four Design in a developing country like South Africa. Learners scheduled their time to 
access CS on the school premises, so it can be concluded that learners developed Self-Regulated Learning (SRL). This study 
is unique because no studies on biodiversity at high school have been reported in the South Africa context. This study 
contributes to encouraging teachers with technology phobia to gain the courage to use ICT to teach science. Teachers in 
developing countries may adopt the use of CS to teach science like their counterparts in the developed countries. This is 
very vital, especially during the COVID-19 pandemic and possibly post COVID-19 era, where virtual learning is inevitable. 

In the developed countries, each learner has a computer to use during studying. It is not the case in developing 
countries like South Africa, where a computer is shared among many learners. While using CS increased learners’ per- 
formances with four to five learners sharing a computer, it is unclear how many learners per computer will be needed 
before compromising the positive effects. The effect of many learners sharing one computer was not investigated but 
may interest researchers. Also, a comparative study of the use of CS between the developed and developing countries 
needs further studies. The limitations of the study were: 1) few learners 33 per group may not be a true representation 
of the population, and thus there is a need for studies with larger samples for more reliable results; 2) learners were 
from one geographical region which may have limited the level of performance; 3) the simulations could not be done 
individually due to lack of home computers; and 4) the time for the study was short, which may have disadvantaged 
slow learners. 


Conclusions and Implications 


Learners taught using CS performed better than their counterparts taught using TCM. It has been established that 
CS are effective tools for enhancing learners’ performance. The study confirmed CS did not discriminate against gender 
because both males and females in the EG performed equally well, but not learners taught using TCM. These results 
imply that the Department of Education should equip schools with computers and build teachers’capacities to embrace 
simulations in science teaching. When learners are provided with computers, they can develop interest and autonomy 
in science learning and this is relevant when considering the online teaching in the New Normal during and after the 
COVID-19 pandemic. These results may provoke other researchers to extend the study by involving larger samples 
from diverse geographical representations and use simulations in different subjects and topics. In addition, teachers’ 
attitudes towards CS and a comparative study regarding the effect of learner-computer ratio may need further study. 
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Abstract. Today science teachers are 
expected to understand the nature of en- 
vironmental education, and be competent 
in skills, methods, and procedures relevant 
to a science discipline. The purpose of 

this research was to explore pre-service 
science teachers’ views with regard to 
environmental education, investigation 
conducted during a freshwater ecosystem 
studied, skills developed and the associ- 
ated stepwise scientific process. Data were 
collected through a five-item questionnaire 
that included the characteristics of envi- 
ronmental education, types of scientific 
investigations, science process skills and 
the scientific process. 94 students that reg- 
istered for a Bachelor of Education degree 
participated in this research. The results 
from statistical analysis of the teachers’ 
responses showed the importance of life- 
long learning, involvement of community 
in environmental education and devel- 
opment of critical and problem-solving 
skills. However, most teachers could not 
provide correct scientific investigation 
they conducted during the freshwater 
study. Although there were no correlations 
between the investigation, science process 
skills and the associated scientific process, 
influence of the stepwise scientific process 
recommended in the science curriculum 
was evident in the teachers’ linking of 
form a hypothesis to observation. The 
findings imply that teachers face potential 
challenges in understanding of scientific 
investigations and the scientific process. 
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Introduction 


Environmental education (EE) had long advocated a multidisciplinary 
view of an environment, acquisition of skills, knowledge, values, attitudes 
and solving of environmental problems (UNESCO, 1980). Today programmes 
related to EE could also be viewed as instrumental in the acquisition and 
development of skills, affective domain, and human-ecosystem connec- 
tions (Powell et al., 2019). This is reasonable because explanations based 
on change in natural systems on earth are key in the understanding of the 
scientific concepts, processes, and tracing of the origins of environmental 
issues (Anand, 2013). Amid the issues thereof, EE has since presented a new 
challenge to teachers (Reddy, 2011), hence a need for the development of 
their expertise and enthusiasm (Lock, 2010). Thus, it can be argued that ac- 
quisition, for instance, of skills, understanding of scientific concepts and of 
the characteristics of EE, need to be prioritised and developed in appropriate 
and/or relevant settings to enhance their retention by teachers. Dresner et 
al. (2014) suggested that with regard to learning and retention of knowledge 
of the ecosystems, laboratory work and lectures may be supplemented with 
experiences that are based on field-based research and modelled the scientific 
process. This argument is reasonable considering that the scientific process 
is a model to problem solving, particularly in science. It melds with science 
process skills (SPS), which according to Gultepe (2016) are considered key 
to science education. 

In EE there have been methods and processes to support change- 
oriented learning towards better environmental sustainability practices and/ 
or environmental learning in a wide range of contexts in South Africa (Fundisa 
for Change Programme: FCP, 2013; Rosenberg et al., 2008; Shava & Schudel, 
2013; Vogel et al., 2013). The learners studying science are expected to criti- 
cally show responsibility towards the environment and also develop scientific 
skills and processes associated with investigating natural phenomena (De- 
partment of Basic Education: DBE, 2011). Sadly, teachers who are entrusted to 
develop learners’ educational outcomes remain part of education problems 
in South Africa (Spaull, 2019). In relation to EE, there has been emphasis on 
the capacity of teachers to implement it (EE) in the curriculum (Reddy, 2011; 
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FCP, 2013). One would expect such teachers to have competence, for instance, in the knowledge, skills, values, prin- 
ciples, methods and procedures relevant to disciplines (Molefe et al., 2016) such as EE. For these teachers to show 
such competence, they should understand and value scientific inquiry (J. Lederman et al., 2018). They should also 
show interest in science and environmental awareness, as well as proficiency in scientific knowledge (FCP, 2013). 
Most importantly, they should understand that fieldwork is a signature pedagogy for future outdoor EE teachers 
(Thomas & Munge, 2017), as it also accommodates scientific inquiry (Remmen & Frayland, 2014). 


Literature Review 


The most influential work related to EE can be traced as far as the Tbilisi Conference (Reddy, 2011; Charoen- 
silpa et al., 2012). EE was then envisaged as an action-based tool of teaching in which development of awareness, 
knowledge, skills and attitudes “assume[d] their full significance with the problems of the environment” studied 
through excursions, field work, and study trips (UNESCO, 1980, p. 44). Years on, an emphasis has also been on EE 
research and the associated study of ecological and issue-related scientific knowledge, empowerment of learners 
and study of curricula material (Scott, 2009). In other words, EE may be viewed as a key to environmental aware- 
ness - a phenomenon supported by Littledyke (2008), Moyo and Masuku (2018) and Soto-Cruz et al. (2014). Fur- 
thermore, it (EE) may point to the synergy between inquiry-based learning and ecology (Taylor & Bennett, 2016) 
as well as the scientific process (Tang et al., 2009; Thomas, 2012) within the context of science education. Finally, 
it may involve ecology-based training for a pre-service teacher (GUlUm, 2011) and thus development of ecology- 
based content and SPS (Colley, 2006). 

It should be noted that the quality of teachers determines that of an education system (Spaull, 2019). Therefore, 
it is reasonable that practice of EE showed that the associated programmes that support teachers are key (Eames 
et al., 2008; FCP, 2013; Reddy, 2011). Thus, EE should be part of science teacher's education for its (EE) characteris- 
tics to be enshrined in science education. That said, the definition of the environment itself has long evolved and 
can now be explained in broader terms (Reddy, 2011). Teachers need to understand the multidisciplinary nature 
of the environment before they engage in EE. EE itself entails environmental knowledge, which in turn encom- 
passes knowledge and awareness about environmental issues the world is confronted with today as well as the 
solutions (Boca & Saracli, 2019). Thus, the researchers contend that in science education it is important that prior 
to blending EE with scientific investigations, the scientific process and the associated SPS, teachers’ views about 
its (EE) characteristics are illuminated. Sondergeld et al. (2014) argued that EE should not only integrate multiple 
content areas but also make education relevant, use social context and promote lifelong, forward-looking educa- 
tion. The proposition of lifelong education points to one of the initial characteristics of EE that teachers should 
understand. The other characteristics include integration of education into community and the interdisciplinary 
and holistic nature of EE even in its application (Molefe & Aubin, 2021). Its approach to solving problems (another 
characteristic) dovetails with Ntanos et al’s (2018) arguments about EE benefits in terms of critical thinking skills 
and problem-solving skills. 

Fieldwork remains a long-standing pedagogy across a range of disciplines in higher education institutions 
(HEls) (Thomas & Munge, 2017). It is considered one of the leading mechanisms for teaching and learning and 
doing science (Allen, 2014). It may also be a relevant method in an “ecosystem study...a water pollution test, a 
biodiversity audit and general observations...” (Rosenberg et al., 2008, p. 2; emphases added) that may be central 
to enabling pre-service teachers make connections between the domain of observable(s) and that of ideas. The 
implication is that teaching about ecosystems might provide an opportunity for inquiry learning (Taylor & Bennett, 
2016). This explains why the popularity of fieldwork is rooted in its ability to accommodate inquiry-based learning 
in which students may engage in scientific investigations (Remmen & Frgyland, 2014), understand ecology con- 
tent knowledge (Colley, 2006) such as that related to biodiversity and pollution (Shava & Schudel, 2013) and SPS 
(Molefe et al., 2016). In other words, by incorporating scientific inquiry in fieldwork in EE, teachers are first helped 
to engage in scientific investigations, depending on types or research questions (N. G. Lederman et al., 2013). Sec- 
ondly, the teachers may be able to apply scientific knowledge in interdisciplinary context. Thirdly, drawing from 
Ozdem-Yilmaz and Cavas (2016), teachers may also have an opportunity to consolidate scientific processes with 
scientific knowledge, scientific reasoning and critical thinking to advance scientific knowledge associated with EE. 
Educators’ understanding of scientific inquiry and scientific investigations remains relevant even today. This is even 
more important considering that the issue of the scientific method and SPS are enshrined in these two concepts (J. 
Lederman et al., 2018). Thus, it can be argued that the inclusion of scientific investigations in the present research 
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was important because teachers’ development of SPS might be necessitated through them (scientific investiga- 
tions). The success of the development of SPS may form the basis for their understanding of how SPS blend with 
the scientific process within an equally understood context - EE. 

The role of teachers in their students’ development of SPS cannot be overemphasised (Saban et al., 2019). 
Literature review shows that while SPS remain the key part of research over the last 10 years (e.g., Gultepe, 2016; 
Molefe & Stears, 2014; Silay & Celik, 2013; Tilakaratne & Ekanayake, 2017; Yakar, 2014), they (SPS) had long been 
central to debates on processes and content (Millar & Driver, 1987; Wellington, 1989; So, 2003). Molefe et al. (2016) 
have elucidated on the debates around teaching and/or development of SPS, conceptual understanding and 
context. Gultepe (2016) contended that as teaching science encompasses the content and processes and skills, 
“underestimating content over process or process over content is unacceptable”, as both are equally important (p. 
780). As referred to earlier, SPS (and thinking or critical thinking skills [Foskett, 2000; Taylor & Bennett, 2016]) can 
be part of a study of ecosystems (VanLeuvan & MacDowell, 2000). Despite criticisms around SPS, their develop- 
ment may improve teachers’ understanding of environmental concepts today (Cf. Irwanto et al., 2018). Use of an 
integrated learning strategy may be effective in developing teachers’ SPS and awareness of the river environment 
(Winarti et al., 2018). Thus, SPS should be linked to EE and engaged in and developed subsequent to conceptual 
understanding such as understanding of environment in a broader sense using a suitable teaching and learning 
strategy. Most importantly, they should be linked to the scientific process itself. 


Frameworks for EE, Scientific Investigations, SPS and the Scientific Process 


Suitable frameworks were needed in the present research because there are various views about concepts 
used such as EE, scientific investigations, SPS and the scientific process (scientific method). Indeed, literature has 
different lists of the characteristics of EE (e.g., Baez et al., 1987; Charoensilpa et al., 2012; UNESCO, 1980), SPS them- 
selves (e.g., DBE, 2011; Gultepe, 2016; Molefe et al., 2016; Saban et al., 2019), science processes (DBE, 2011; Moeed, 
2013; So, 2003; Watson & James, 2004), as well as scientific investigations (Moeed, 2013; So, 2003; Watson et al., 1999). 

For teachers to be able to explain natural phenomena scientifically, they should have an environmental aware- 
ness (e.g., of the nature of EE). They should not only know the goals and purposes of science investigations but 
also the types of scientific investigations (Moeed, 2013; Watson et al., 1999). In order to contextualise the present 
research, the researchers drew from UNESCO (1980), Baez et al. (1987) and Watson et al. (1999) to devise frame- 
works that might act as a lens for this research with regard to the characteristics of EE and the types of scientific 
investigations (Table 1). 


Table 1 
The Nature of EE and Types of Scientific Investigations 


Characteristic features of EE Types of scientific investigations 
1. Itis a life-long, forward-looking education. (1) Fair testing and comparing 
2. Itinvolves integration of education into the community. (2) Pattern seeking 
3. Itis interdisciplinary and holistic in nature and its application. (3) Classifying and identifying 
4. It encourages the development of sensitivity, awareness, under- (4) Exploring 


standing, critical thinking and problem-solving skills. 


As humanity ventures into the 4" Industrial Revolution with its glaring grand challenges, the importance of 
SPS and environmental awareness cannot be overemphasised, especially in teacher education. Sadly, there are 
also misconceptions about the associated scientific process, and fair testing is at the centre of them (Moeed, 2013; 
DBE, 2011). Thus, the misconception around fair testing needed to be diagnosed and corrected. For a framework 
on skills and the scientific method, they drew from SPS, and the stepwise scientific process stipulated in the South 
African Natural Sciences curriculum (DBE, 2011). Furthermore, they drew from Molefe and Stears (2016) and Watson 
and James's (2004) ideas to show how such skills may blend into those scientific processes (Figure 1). 
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Research Aim and Research Questions 


The aim of this research was to explore pre-service teachers’ views in relation to the characteristics of EE it- 
self, scientific investigations, SPS they developed and the associated scientific process. Many studies analysed by 
Alvarez-Garcia et al. (2015) have pointed to teachers’ indisputable role in “the infusion of EE into schools as a tool 
to environmentally educate future citizens” (p. 81). Teachers have also been at the heart of empirical work on biol- 
ogy field work for decades (Lock, 2010; Tilling, 2018). Thus, the present teachers were a perfect fit for the research. 
They studied EE in their Life Sciences module offered at an institution in KwaZulu-Natal, the components of which 
comprise annual three-day fieldwork in which different ecosystems that include freshwater are investigated. The 
research has focused on the freshwater ecosystem because it enabled the researchers explore all the key aspects 
they sought to study. 

The research reported here has been more imperative than ever. Global problems and threats in our time 
include the environment and climate (Ivanov, 2018). Thus, it is important that scientific literacy, interest in science 
and inquiry-based instruction are incorporated in programmes intended to increase school students’ environmental 
awareness (List et al., 2020). Freshwater is part of a nexus that is proposed as a tool to transform human well-being 
in Southern Africa (Mabhaudhi et al., 2019; Mpandeli et al., 2018). This is reasonable because about 60% of water 
used in, for instance, mining and the associated mine dumps and landfills impacts on freshwater, hence biodiversity 
(Department of Environmental Affairs, 2018). Furthermore, it is through EE in higher education that future teachers 
could be prepared for a green society today (Boca & Saracli, 2019). Most importantly, this research was part of a 
project that investigated pre-service teacher learning within science and technology education modules (Molefe et 
al., 2017), and our latest work on SPS and the scientific process intertwined with global environment issues today. 


Figure 1 
The Scientific Process and the Associated Science Process Skills 


Problem/Research 
Question 


Observing 
Communicating 


Form a hypothesis 
Communicating 


Conclusion & 
inferences 


Interpreting Informati Predicting 
Communicating Planning investigations 
The Scientific Process Hypothesising 


Science Process Skills 
(DBE, 2011) 


Observation 
Communicating 
Observing 
Measuring 

Sorting & classifying 


Design activity/experiment 
Measuring 
Communicating 
Planning investigations 


Note. Adapted from Molefe and Stears (2016) 
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The research sought to answer the following research questions in relation to an EE module: 


e What characteristic features of EE were embedded in the pre-service science teachers’ fieldwork? 

e What type of scientific investigations were conducted by the pre-service science teachers during their 
freshwater study? 

e What correlations are there between the teachers’ views of a freshwater ecosystem in terms of scientific 
investigations, processes and the associated SPS used? 


Research Methodology 
General Background 


This research adopted a quantitative research approach. The approach allowed the researchers to utilize survey 
design (Neuman, 2014). Survey designs normally provide a numeric description, for instance, of trends or opinions 
of a sample of a particular population (Creswell, 2014), which include correlations among variables (Cohen et al., 
2018). In this research, a survey administered was descriptive and analytic in relation to the teachers’ views about 
EE, scientific investigations, SPS and the scientific process within the context of the freshwater study. A question- 
naire was employed because it is commonly used in SPS-based studies (Fugarasti et al., 2019). Furthermore, it was 
utilised in this research because it could elicit teachers’ views about scientific processes and skills (Coil et al., 2010) 
and scientific investigations (Moeed, 2013). It could also be used to study learners’ understanding of scientific 
inquiry (J. Lederman et al., 2018). 


Participants 


Data were collected from 94 pre-service teachers who were registered for a second-year Biological Sciences 
Education module at the institution. The sample size gave a fairly good reliability because the minimum one (size) 
that could be used for some statistical analysis is 30 (Cohen et al., 2018). Nevertheless, it should also be noted that 
this conveniently selected group of teachers completed the questionnaire as part of their fieldwork exercise. Thus, 
the sample selection might have weakened the research’s external validity. The research questions and design of 
the research were not based on the demographics of the teachers such as those related to gender and age. Thus, 
these variables were not included. Coil et al. (2010) pointed to the importance of a scaffolding approach and 
iterative practice, particularly in relation to SPS. The approach and the practice were essential before the present 
teachers could provide views about EE, scientific investigations, SPS and how they meld with the scientific process 
steps. Thus, the teachers were suitable participants because they had at least 18 months’ exposure to SPS during 
the relevant method and content module classes and the associated practical activities. These teachers were as- 
sured of absolute anonymity. 


Instrument and Procedures 


A five-item questionnaire created by the researchers was used to establish the teachers’ views in relation, for 
instance, to EE itself, scientific investigations, the scientific process, and development of SPS. The questionnaire, 
with attached copies of the detailed description of SPS and the scientific process steps (DBE, 2011), and information 
about EE and scientific investigations, had qualitative and quantitative components. Questions one and two of the 
questionnaire provided qualitative data on activities the teachers found interesting with regard to the ecosystems 
they studied, and the descriptions of EE characteristics they learnt, respectively. The third, fourth and fifth ques- 
tions provided quantitative data on scientific investigations, SPS, and SPS and the scientific process. In elaboration, 
with regard to the last three questions, the teachers were expected to select the types of scientific investigations 
they thought they used during the freshwater study. Second, they selected five SPS used to deduce the quality 
of the freshwater ecosystem studied. The teachers also provided activities that developed the SPS thereof. Third, 
they selected two SPS that they viewed to fit into the scientific process steps when investigating the freshwater 
ecosystem using the chemical test kits. The research explored correlations between the teachers’ responses to the 
questions except the first two. 
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Survey research is prone, for instance, to measurement and nonresponse errors (Ponto, 2015). Thus, piloting 
a questionnaire might be useful in addressing issues related, for instance, to clarity and elimination of questions’ 
ambiguities (Cohen et al., 2018). In this research, a pilot test of the questionnaire was conducted with a tutor and 
ten demonstrators who successfully completed the module. It should also be noted that Molefe and Stears’s (2016) 
findings on SPS, the scientific process and the scientific inquiry were further used to improve the questionnaire 
for the research. In addition to the improvements made (e.g., the questionnaire sectionalization and itemization), 
the results of the pilot test were used to revise the questions of the questionnaire where necessary. For example, 
investigations were reworded into scientific investigations. The teachers were further requested to provide details 
concerning SPS they developed during the freshwater study (see the questionnaire’s Table 2) and an example was 
given to enable them to successfully complete the last question on the questionnaire (see the questionnaire’s 
Table 3). 

The questionnaire was administered during a fieldwork period. Ethical clearance for the project investigat- 
ing pre-service teacher learning within science and technology education modules at the institution was used. 


Data Analysis 


Cohen et al. (2018) pointed to the importance of data entry and cleaning. Such processing of data was 
conducted using OpenRefine. The data required a statistical package that could enable the researchers to com- 
pute descriptive and analytical functions essential to answer the research questions. R was used because of its 
extensive application in statistical and graphical techniques. For questions concerning EE’s characteristic features 
and type of investigation performed by the teachers, the researchers needed to verify that the answers were not 
given randomly. Thus, the preliminary results were based on Chi-squared test on the frequencies. Then, Kendal and 
Pearson correlation tests were used to measure the degree of the relationship between variables of the questions. 
The tests were corrected using a Bonferroni correction. 


Research Results 


This research sought to illuminate the teachers’ views of the characteristic features of EE (EECF) and scientific 
investigations prior to exploring those related to SPS and the scientific process. It was important that a test was 
made to check whether the answers concerning these two variables were given randomly. Chi-squared test was 
used to compare the given answers to uniform (random) ones. The respective p-values (Table 2) show that the 
answers for EECF and scientific investigations were not given randomly. 


Table 2 
p-values of Chi-squared test for uniformity for EECF and scientific investigations 


Variables y2 df p-value 
EECF 32.9 4 < .001 
Investigations 57.5 5 < 001 


Characteristic Features of EE that Were Embedded in the Pre-Service Science Teachers’ Fieldwork 


One of the activities that were part of the teachers’ fieldwork was a collaborative development of critical think- 
ing skills within the context of nature conservation and sustainability. Hereafter, they were expected to describe 
any two EECF (Table 1) they had developed before the researchers assessed, for instance, their SPS and values 
essential for solving environmental problems today. 

The uniformity test was followed by further exploration of the dataset. Table 3 shows that the following pair 
of EECF were given most often: /t is a life-long, forward-looking education and it involves integration of education into 
the community. These characteristics together with /t encourages the development of sensitivity, awareness, under- 
standing, critical thinking and problem-solving skills resulted in the second and third most given pairing, respectively. 
Some of the written descriptions associated with the results were: 
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Table 3 
Frequency Table for EECF 
EECF Frequency 

It is a life-long, forward-looking education and it involves integration of education into the community. 29 

It is a life-long, forward-looking education and it is interdisciplinary and holistic in nature and its application. 5 

It is a life-long, forward-looking education and it encourages the development of sensitivity, awareness, understanding, critical 97 

thinking and problem-solving skills. 

It involves integration of education into the community, and it is interdisciplinary and holistic in nature and its application. 2 

It involves integration of education into the community, and it encourages the development of sensitivity, awareness, under- 4 

standing, critical thinking and problem-solving skills. 

It is interdisciplinary and holistic in nature and its application, and it is interdisciplinary and holistic in nature and its application. 3 


EECF 1 (see Table 1): /t [EE] is always being updated and the information added because new species. ..are forever discov- 
ered by people and studied (Teacher 1). 

EECF 2: This involves... people taking responsibility of the environment that they live in. This can be done through summits, 
and programmes could arise that teach people about conservation of nature and sustainability in the community (Teacher 2). 
EECF 4: Lectures give us platform to gather information for ourselves. ..This develops and stimulates [sic] our problem-solving 
and critical thinking skills... Fieldwork taught us awareness and sensitivity to nature; to value every species... (Teacher 3). 


Scientific Investigations that Were Conducted by the Pre-Service Science Teachers during Their Fieldwork 


The freshwater study was intended to build on practical and theoretical emphases on scientific investigations 
done at the institution. The present teachers’ reflections on scientific investigations (Table 1) done enabled the 
researchers to explore this. The results (Table 4) showed that Classifying and identifying (92), Exploring (73) and Fair 
testing and comparing (62) were given most often. 


Correlations Found Between the Teachers’ Views of a Freshwater Ecosystem in Terms of Scientific Investigations, 
Processes and the Associated SPS Used 


Teachers should be primary creative and critical thinkers concerning solutions to societal issues today. Hence, 
investigating a nexus of scientific investigations, the scientific process steps and SPS was imperative. The researchers 
acknowledge that, similar to EECF and scientific investigations, it was important that they presented their find- 
ings concerning SPS and the scientific process before those related to correlations between these two variables 
thereof and the investigations. Nevertheless, it should be noted that this research was a follow-up to one in which 
they (researchers) studied the present teachers’ views about how SPS blend with the scientific process (Molefe & 
Aubin, 2021). Thus, in this research they chose to focus solely on correlations between several variables (Table 5). 


Table 4 
Frequency Table for Scientific Investigations 


Types of scientific investigations Frequency 
Fair testing and comparing 62 
Pattern seeking 31 
Classifying and identifying 92 
Exploring 73 
Making things or developing systems 6 


Note. The students could give more than one answer. Only the marginal frequencies of the given answers were considered. 
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The researchers computed Kendal’s rank correlation tau (t) and Pearson's product-moment correlation (r) 
to determine the relation between the variables. The results of the Kendal correlation showed that there was a 
significant positive association between Form a hypothesis and Observation (t = 3.28, p = .001). Pearson correlation 
also indicated the same association between the two variables thereof, which was also statistically significant (r= 
3.53, p= .001). This means that correlations between the number of incorrect answers related to Form a hypothesis 
and Observation were significatively greater than zero. 

In relation to SPS, the researchers drew a bar plot of the incorrect answers (i.e., answers far from Figure 1) 
given by the teachers (Figure 2). The results showed that all the teachers provided less than two incorrect answers 
out of five answers related to the SPS observing, comparing, recording information, and communicating, which is a 
good result. For random answers, they computed the same bar plots of incorrect answers (Figure 3). 

The results indicated higher values (from 1 to 5), which shows that answers given by the teachers were clearly 
better than random answers. The researchers then tested for correlations between incorrect answers given by the 
teachers and those (number of incorrect answers) concerning the other variables (Table 5). The findings could not 
show any correlations. 

In relation to scientific investigations, the findings could not show significant correlations between the number 
of incorrect answers given by the teachers and the number of mistakes concerning the other variables. 


Discussion 


The concept of EE (Baez et al., 1987; Reddy, 2011; UNESCO, 1980), development of skills and knowledge con- 
cerning the environment (Colley, 2006; VanLeuvan & McDowell, 2000) or using environment (and ICT) (Osman & 
Vebrianto, 2013), the scientific investigations (Watson et al., 1999) and teachers’ perspectives thereof (Moeed, 2010) 
and controversy around the scientific process (Thomas, 2012; Watson & James, 2004) are hardly modern. Never- 
theless, EE in tandem with sustainable development have become part of recent key debates on environment at 
local (Mabhaudhi et al., 2019; Mpandeli et al., 2018) and international levels (Boca & Saracli, 2019). Research over 
the last five years has also focused on SPS and achievement (Prayitno et al., 2017), SPS assessment using scenario 
based MCQs (Temiz, 2020), understanding of SPS (Shahali et al., 2017), phases of inquiry (Pedaste et al., 2015) 
and inquiry skills by future biology teachers (Cipkova & Karoléik, 2018), context and the debates around teaching 
and/or development of SPS, and conceptual understanding and context (Molefe et al., 2016). However, teachers’ 
views of the freshwater ecosystem in terms of EE, scientific investigations, processes and the associated SPS they 
used remains a domain that is hardly been ventured into. This research sets a precedent of aspects that another 
research may learn from. 


Table 5 
Correlation Matrix for the Scientific Process, SPS and Scientific Investigations 


Research Formulating Designing Observations Conclusions seeaesle cae 

question (ReQ) hypothesis (FoH) experiment (DeE) (Obs) (Con) (SPS) (Inv) 

ReQ 1.000 -.212 -.004 -.121 -.059 011 045 
FoH -.212 1.000 .038 845 -.020 .086 207 
DeE -.004 038 1.000 -.212 224 -.062 037 
Obs -.121 345 -.212 1.000 -.019 -.080 -.145 
Con -.059 -.020 224 -.019 1.000 024 -.131 
SPS 011 .086 -.062 -.080 024 1.000 -.055 
Inv 045 207 .037 -.145 -.131 -.055 1.000 


Note. The Bonferroni correction, .05/21 = .002, enabled the researchers to reject a null hypothesis of randomness if the p-value 
was less than .002. 
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Figure 2 


Bar Plot of Answers from the 94 Teachers 
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Figure 3 
Bar Plot of Random Answers from the 94 Teachers 


35 


30 


P ii. 


Number of incorrect answers 


Frequency 


EE might be instrumental in the delivery of various benefits related to natural ecosystems (West, 2015). An en- 
vironment itself can be used to develop SPS (Osman & Vebrianto, 2013). Thus, it can be argued that studies related to 
natural ecosystems should include investigations using SPS. Students’ ecosystem-based worksheet might not only 
be practical, but it could be used to enable them to develop SPS (Patresia et al., 2020). The present teachers engaged 
in a fieldwork where activities included the use of a worksheet and several resources such as bug dials to investigate 
the quality of the freshwater ecosystem. They were also challenged to reflect on characteristics of EE they developed 
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and the scientific investigations they employed as the basis for development of SPS using a stepwise scientific process. 

The present teachers viewed EE as alife-long, forward-looking education that involves integration of education into 
the community. They went further to pair these characteristics of EE with /t encourages the development of sensitivity, 
awareness, understanding, critical thinking and problem-solving skills. 

Extensive literature reviewed by Alvarez-Garcia et al. (2015) pointed to challenging issues around environmental 
competences and the associated teacher training. The researchers were conscious of such challenges in the pres- 
ent research. Thus, the freshwater study was set to nourish life-long environmentally literate habits such as outdoor 
experience and collaborative work on meaningful environmental issues (Sondergeld et al., 2014). They also tapped 
into the teachers’ critical thinking skills in terms of nature conservation and sustainability in tandem with communal 
and ecosystem-based impact of mining where they conducted their fieldwork. The associated activity also tapped 
into their sensitivity, awareness and conscience and reinforced the theoretical EE aspects developed at the institution. 
The impact, for instance, of critical thinking (Arslan, 2012; UNESCO, 1980), the need to raise awareness of sustainable 
development and understanding of environment in a broader sense (Reddy, 2011) had long been associated with 
EE. The results of the present research may thus be understood with regard to guided inquiry and problem-solving 
processes as the means for life-long learning (Burbules et al., 2020), the importance of development of skills of life- 
long learning in promoting sustainability (Yli-Panula et al., 2020), critical thinking and problem-solving as key aspects 
in sustainability and for future success and survival (Taimur & Sattar, 2020) and the need to raise awareness about 
key issues facing the planet today. 

The importance of an iterative approach to science investigations and the associated SPS, conceptual under- 
standing and reflexivity cannot be overemphasised (Chirikure, 2019). It is thus reasonable that there has long been 
a call for research on understanding of scientific inquiry rather than mere “doing” of it (inquiry) (N. G. Lederman et 
al., 2013). The experience of developing SPS during the freshwater study was meant to provide the teachers with a 
framework and understanding of scientific investigations. The results showed that Classifying and identifying, Exploring 
and Fair testing and comparing dominated the selections made by the teachers. It should be noted that the teachers 
drew from observed indicator species to deduce the state of the freshwater ecosystem they studied. They captured, 
identified, and classified those species according to their sensitivity to pollution using resources provided. 

An investigation phase of inquiry may be characterised by “explore” or“exploration” (Harlen, 2014; Pedaste et al., 
2015). It should be noted that understanding scientific inquiry itself has been a challenge for teachers and learners (J. 
Lederman et al., 2018). Exploring (Table 4) involves making careful observations over time (Watson et al., 1999, 2006). 
Exploring might not be as common as fair testing and classification in the later years of an education system (Moeed et 
al., 2016). Thus, the wrong selection of Exploring by the present teachers may be understood in terms of investigating 
(rather than making observations of) indicator species to deduce the state of the freshwater ecosystem studied. This 
finding, by virtue of being the second most selected type of scientific investigation, requires further research. On the 
other hand, the incorrect Fair testing and comparing may be understood in terms of emphasis on scientific investiga- 
tions in the South African Natural Sciences curriculum that are tailored to fair testing (DBE, 2011). Bias towards fair 
testing is also found in Hume and Coll (2010) and Moeed (2010, 2013). The present teachers’ mixed views concerning 
the scientific investigation they developed suggest possible lack of or limited experience of scientific investigations 
in high school (Kazeni et al., 2018; J. Lederman et al., 2018) and/or at tertiary level (Molefe & Stears, 2014). That might 
have, in turn, impacted on their understanding that there is no linear approach to solving problems, especially those 
related to the environment in a broader sense. The researchers believed that such a deficiency might also impact on 
the teachers’ conceptual understanding of SPS and the scientific process, both of which are equally emphasised in 
the curriculum. But there was no correlation between incorrect answers given by the teachers on scientific investiga- 
tions, SPS and the scientific process. 

Teachers’understanding of scientific processes might be a fundamental facet of thinking synonymous with science 
and EE-a reason SPS could also be considered as life-long learning skills (Temiz, 2020). Molefe and Aubin's (2021) results 
from statistical analysis of the present teachers’ responses to skills they developed during freshwater study showed 
prominence of observing. This is a SPS that has been rated highly at the institution (Molefe et al., 2016). Furthermore, 
teachers might be successful in identifying the skill (Gultepe, 2016). In relation to the stepwise scientific process, the 
teachers’ responses showed a good proximity to the expected representation of the scientific process model (Figure 
1) with regard to Form a hypothesis. On the other hand, Observation showed a relatively close proximity to the model. 

As referred to earlier, answers given by the teachers concerning SPS were better than random answers (Figure 3). 
Teachers’ perceptions of understanding of integrated SPS such as hypothesising, might be high (Hafizan et al., 2012) 
contrary to their conceptual knowledge (Mumba et al., 2018). That said, there was no correlation between incorrect 
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answers given by the present teachers concerning SPS and those concerning the other variables, including the 
scientific process steps (e.g., Form a hypothesis and Observation). A significant correlation was only evident between 
Forma hypothesis and Observation. Thus, the correlation merely enabled us to understand the direct link between the 
incorrect answers given by the teachers in relation to the scientific process steps. First, the link between the two steps 
may be understood in terms of the scientific process being viewed as making “observations to test the hypothesis and 
drawing a conclusion in support or otherwise of the tested hypothesis” (Moeed, 2013, p. 541; emphases added). The 
association of the two steps is also evident in the Natural Sciences curriculum (DBE, 2011; also see Figure 1). Teach- 
ers are expected to make inferences from their observations, make some conclusions, with their hypothesis in mind. 

SPS have been extensively researched in literature. The popularity of SPS today is reasonable because they are 
not only the life-long learning skills, but their development is a fundamental goal of science education (Tamiz, 2020). 
They may enable teachers to not only understand information better, but also develop their critical thinking skills 
and decision-making (Hafizan et al., 2012). The environment remains an outdoor laboratory to develop them (SPS). 
Yet, research on their (SPS) association with scientific investigations, the stepwise scientific process and EE is very 
limited. Buchanan et al. (2019) argue that teachers need to make connections to the scientific method, real-world 
issues, learning and action when engaging with digital technologies for environmental purposes. Their argument, 
coupled with the current research’s findings, suggest a sustainable future that will disrupt the notions of the science 
education curriculum. 

Lortie’s (1975) long-standing views about teacher educators’ praxis and its impact on teachers’ learning through 
own “apprenticeship of observation” have now assumed even greater importance. The present findings imply that 
teacher training should offer apprenticeship that embodies development of life-long skills and 21* century skills. The 
training should also value integration of education into the community. 

It was interesting that there was only a link between two scientific steps - Form a hypothesis and Observation. 
Students might be confronted with some challenges when conducting scientific investigations (Ramnarain, 2011). 
Indeed, the present research suggested an influence of science curriculum on teachers’ understanding of the scientific 
process and scientific investigations. Teachers need to understand the fundamental aspect of scientific investigations, 
that is, our observations of natural phenomena (and scientific investigations) are inspired and guided by problems 
or questions (N. G. Lederman et al., 2013). Therefore, not all scientific investigations would be based on formulated 
hypothesis as they (investigations) might be solely rooted in, for instance, the Classifying and identifying type (Table 1). 
For teacher educators, the findings pointed to a need for new ways in which they might better enable their students 
to reconcile the understanding of types of scientific investigations, SPS and stepwise scientific process. 


Conclusions and Implications 


The research showed that EE might be viewed as a life-long, forward-looking education that involves integration 
of education into the community. The research further pointed to the importance of the development of sensitivity, 
awareness, understanding, critical thinking and problem-solving skills in relation to the two characteristics of EE thereof. 

Classifying and identifying, exploring and fair testing and comparing were the common types of scientific 
investigations selected by the teachers although they simply observed, identified, and classified indicator species 
according to their sensitivity to pollution in the freshwater ecosystem studied. The results, particularly in relation to 
fair testing, indicate that the teachers did not succeed in providing the scientific investigation they employed. The 
results further show a significant relationship between the teachers’ ability to formulate a hypothesis and observation. 

In this research, it was concluded that EE within the context of teacher education should be characterised by 
skills development and integration of education into the community. Scientific investigations (and the scientific pro- 
cess) should be part of EE-based teacher education, particularly because teachers might have misconceptions about 
them. Thus, there is an urgent need for educational reform in higher education in relation to scientific investigations, 
pre-service teachers’ understanding of SPS and how they (SPS) meld with the stepwise scientific process. 

The present findings make a small but significant contribution to science education in terms of debates on the 
areas teacher educators should prioritise when investigating pre-service teachers’stance on EE, scientific investigations, 
scientific skills and processes intertwined with possible challenges related to the planet's natural ecosystems today. 
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BIOLOGICAL SCIENCE FOR EDUCATORS 


EDUCATIONAL OUTCOMES DEVELOPED DURING LECTURES AND AT ISMANGALIZO 


Student Name & Surname 


Student Number 


QUESTION 1: Please (a) describe the activity that you performed at *iSmangaliso that you found the most 
interesting, and (b) give reasons why you chose it. 


Note. *iSmangalizo is a pseudonym for a place where the students participate in annual three-day fieldwork based on five eco- 
systems (i.e., Freshwater, swamp mangroves, dunes, natural forest, and plantations) and nature conservation and sustainability 
activity. 


QUESTION 2: The Nature of Environmental Education 


Characteristics of Environmental Education 


t is a life-long, forward-looking education. 


t involves integration of education into the community. 


t is interdisciplinary and holistic in nature and its application. 


it encourages the development of sensitivity, awareness, understanding, critical thinking and problem-solving skills. 


Please briefly describe any two (2) of the characteristics of Environmental Education stated in the table above. 
Your descriptions should be based on the development of the two selected characteristics during lectures (at 
university) and fieldwork (at iSmangalizo). 
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QUESTION 3: Table 1 below shows different types of scientific investigations. Please tick ( W ) scientific 
investigation(s) that you used during freshwater study at iSsmangalizo. 


Table 1 Types of scientific investigations 


ITEM List Of Types Of Scientific Investigations Tick ( V4 ) 
3.1 Fair testing and comparing 
3.2 Pattern seeking 
3.3 Classifying and identifying 
3.4 Exploring 
3.5 Making things or developing systems 


QUESTION 4: Science process skills in Environmental Education 


Please refer to the attachment on science process skills (SPS). It has SPS that you should have developed 
during the freshwater study. 

Please list any five (5) science process skills (in Table 2 below) that you developed during your fieldwork at 
iSmangalizo. You are also expected to provide an activity that you did to support your choice of the SPS. 


Table 2 Science process skills developed at iSimangalizo 


Science process skill | developed The activity | did to develop this skill 


QUESTION 5: During your freshwater study, you did not use Chemical Test Kits (CTK) for testing water quality 
at the stream studied. If you did, you would have followed the six (6) processes of science in Table 3 below. 

Please refer to the attachment on science process skills (SPS). It has Science process skills (SPS) that you 
would have developed during the activity, now using the Test Kits. 

Complete Table 3 by writing down any two (2) SPS that you would have developed in 5.1 to 5.6. NOTE: I 
have done 5.2 for you as an example. 
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Table 3 Processes of science and the associated SPS 
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Two Science Process skills | would have 


ITEM PROCESSES OF SCIENCE PROCESSES OF SCIENCE 
developed 

5.1 Problem/Research question: Identify a problem and develop a ques- 

tion. What is it you want to find out about the stream? 
5.2 Research/Background knowledge: What is it that you know about Communicating 

the stream from previous investigations/publications? 

Interpreting information 

5.3 Formulating hypothesis: A hypothesis is your idea, answer, or predic- 

tion about what will happen and why, when you test the state of the stream’s 

water. 
5.4 Design an activity or experiment: Activities you to do to test your idea 

or prediction to see if you were right about the state of the stream. 
5.5 Observation: (a) Observe/note changes/reactions (e.g., change in colour 

of the chemical used), and record your observations (e.g., onto a graph, 

table, etc.), (b) look at the results of your activity or experiment, (c) and write 

about what happened. 
5.6 Conclusion: Make inferences about the observations recorded. Make some 


conclusions. What did you find out? Do your results support your hypothesis? 
What did you learn from this investigation? 
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Introduction 


The Program for International Student Assessment (PISA) conducted by 
the OECD in three-year cycles is an international large-scale program that 
measures 15-year-old students’ knowledge and skills in reading, mathemat- 
ics, and science. PISA compares the quality and equity of education across 
countries, and the results obtained are used by educators and politicians in 
determining appropriate education policies and practices (OECD, 2019a). 
In each cycle of the research, a subject is chosen as the main area. Like the 
previous cycles, the PISA 2018 assessment covered the areas of reading, 
mathematics, and science, but the core area was reading literacy. 

In recent years resources related to ICT have become accessible to stu- 
dents both at home and at school and have gained a large place in modern 
society. Due to the importance of ICT, an optional ICT familiarity question- 
naire about ICT availability, ICT skills and ICT use has been included in each 
cycle of PISA since 2000 (Kunina-Habenicht & Goldhammer, 2020). With the 
change in PISA 2015, ICT engagement questionnaire was added to PISA's ICT 
familiarity survey as a new structure and this scale was also included in PISA 
2018.ICT engagement scale has been improved by both intrinsic and extrinsic 
motivation derived from the theory of self-determination (Deci & Ryan, 2000). 
ICT engagement scale has four factors: 1. Interest in ICT which is the intrinsic 
motivation of using products related to ICT (Zylka et al., 2015); 2. Perceived 
ICT competence, students’ ICT-based knowledge and skills that can be used 
to perform ICT-related tasks (Meng at al.,2019); 3. Perceived autonomy in us- 
ing ICT, perceived control and independence of an individual in using ICT; 4. 
Social Relatedness in using ICT, the measure of an individual communicating 
and interacting with others using ICT (Kuger et al., 2016; Zylka et al., 2015) 

ICT engagement scale is a fairly new scale. While reviewing the literature, 
it was noticed that the early studies leading to this scale originated from the 


https://doi.org/10.33225/jbse/21.20.639 


JOURNAL 
OF®BALTIC 
SCIEN CE 
EDUCATION 


ISSN 1648-3898 reins 
ISSN 2538-71 38 joni 


Abstract. Information and Communica- 
tion Technology (ICT) engagement, as 

a multidimensional construct, plays an 
increasingly important role in education. 
The main purpose of this research was 

to explore the effects of ICT engagement 
factors on science performance across 
Singapore and Turkey conditional to the 
sufficient degree of measurement invari- 
ance of ICT engagement scale. The multi- 
group confirmatory factor analysis results 
demonstrated strong factorial invariance 
of ICT engagement scale across Singapore 
and Turkey, so we were able to use ICT 
engagement scale to meaningful and valid 
comparisons between these countries. 
After obtaining measurement invariance, a 
multi-group structural equation modeling 
was used for the comparison of the effects 
of ICT engagement factors on student’s 
performance of science between these two 
countries. While interest in ICT, perceived 
ICT competence and perceived autonomy 
in using ICT have significant positive direct 
effect on science performance in both 
countries, the direct effect of social related- 
ness in using ICT on science performance 

is negative in both Singapore and Turkey. 
Also, when compared with Singapore, the 
effects of all ICT engagement constructs on 
student's performance are higher in Turkey. 
Keywords: /CT engagement, measure- 
ment invariance, multi-group SEM, science 
performance, PISA 2018 
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use of ICT in school and in home (Meng et al., 2019). Researchers obtained mixed results about the effect of using 
computer and technology on students’ success (Agasisti et al., 2020; Chen & Wang, 2013; Eickelmann et al., 2016; 
Lei & Zhao, 2007; Liemb et al.,2014; Notten & Kraaykamp, 2009; Thiessen & Looker, 2007; Xin & Zhou, 2010). For 
example, Thiessen and Looker (2007) showed that intensive computer use in Canada was negative relationship 
with students’ reading performance whereas Xin and Zhou (2010) showed that computer technology use increased 
students’ performance. Using PISA 2012 data from 15 European countries, the study showed that intensive com- 
puter use in homework negatively affected test scores in all areas (Agasisti et al., 2020). It could be argued that the 
first studies about ICT in the literature were about the effect of computer or internet use on success in school and 
in home (Meng et al., 2019). These studies revealed the need for developing an ICT engagement construct. Later 
studies focused on development of construct related to ICT (De Wit et al., 2012; Janneck et al., 2013; Senkbeil & Ihme, 
2017). Zylka et al. (2015) proposed a new ICT engagement scale which was more motivation and meta-cognition 
perspective of ICT related constructs. ICT engagement scale was adapted to the four-dimensional structure in PISA 
2015. In recent studies, ICT engagement has been considered as a multidimensional construct and its effect on 
academic achievement has been explored. Different results were obtained (Cheema & Zhang, 2013; Hu et al., 2018; 
Lee &Wu, 2012; Luu & Freeman, 2011). For example, perceived ICT autonomy had a positive effect on achievement 
(Cheema & Zhang, 2013). According to the study conducted by Lee and Wu in 2012, interest in ICT and perceived 
ICT competence were positive predictors of reading achievement. 

The ICT engagement scale, developed in PISA 2015 and used in PISA 2018, is a relatively new assessment tool. 
There were only two studies (Meng et al., 2019; Ma & Qin, 2021) exploring the validity of this scale across countries. 
It was necessary to explore the measurement invariance of this scale to make valid comparisons between different 
groups (Mata et al., 2020). Measurement invariance is a property that indicates a measurement tool (a question- 
naire in the case of survey research) measures the same concept in various subgroups in the same way (Davidov et 
al., 2014). The measurement issues of the newly ICT engagement scale need to be further explored. This research 
was devoted to explore the effects of ICT engagement factors on performance of science between Singapore and 
Turkey using Multi-group Structural Equation Modeling (Multi-group SEM) analysis. The first step was composed 
of the analyzes for configural, metric and scalar invariance of ICT engagement scale of PISA 2018 across Singapore 
and Turkey using Multi-group Confirmatory Factor Analysis (Multi-group CFA). As the ICT engagement scale pro- 
vided acceptable invariant, effects of ICT engagement factors on students’ science performance were compared 
between Singapore and Turkey in the second step. 


Research Problem 


In this research, Singapore and Turkey were chosen to compare. Students from these two countries completed 
the optional ICT engagement form in PISA 2018. These two countries represent different education systems and 
different success in PISA. Singapore education system can offer a customized curriculum for student groups created 
according to their technical and social characteristics in the same school, whereas Turkey education system offers 
the same curriculum for all students (Kilic Depren, 2020). Singapore and Turkey had different science performance 
in PISA 2018. According to PISA 2018 results, the average science performance of 15-year-olds in Turkey was 468 
points and it was less than 489 points which was the average for OECD countries participating in the same cycle. 
Turkey's change in performance of science between PISA 2015 and 2018 indicated one of the strongest increases 
among participating countries and economies. In Singapore, whose average in science performance was one 
of the highest among countries and economies participating in PISA, the average score was 551 (OECD, 2019c). 
Singapore has national plans for ICT integration and implementation covering the educational environment and 
resources. Turkey’s ICT policy also aims at improving both ICT infrastructure and capacity of stakeholders according 
to “Information Society and Action Plan (ISAP) for 2014-2018" prepared by The Ministry of Development (MoD) of 
Turkey. Considering that both countries are focused on the same goal in terms of ICT, Singapore, with the highest 
science achievement, and Turkey, with the highest increase in science performance between PISA 2015 and 2018, 
were selected to compare the effectiveness of ICT policy. 


Research Focus 


In this research, the measurement invariance of the ICT engagement scale was examined across Singapore 
and Turkey. If a sufficient degree of invariance is achieved (i.e., scalar invariance), then the effects of ICT engage- 
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ment factors on performance of science could be compared across these two countries. Specifically, the following 
research questions were addressed: 
1. To what extent does the ICT engagement scale show invariance across Singapore and Turkey? (Multi- 
group CFA) 
2. Ifa sufficient level of invariance can be established, how do the effects of ICT engagement factors on 
student science performance differ by Singapore and Turkey? (Multi-group SEM) 


Research Methodology 
General Background 


PISA evaluates to what extent 15-year-old students who are about to complete their compulsory education 
have acquired the knowledge and skills necessary for them to adapt to modern societies. This assessment examines 
the extent to which students can draw conclusions from the information they have acquired and to what extent 
they can apply this knowledge in school and out-of-school, rather than whether students can reproduce informa- 
tion. The PISA 2018 evaluation is the seventh since the program started in 1997. In the program, which is carried 
out and developed in cooperation between the governments of OECD countries and partner countries/economies, 
evaluations focusing on basic areas such as reading, mathematics and science are carried out in three-year cycles. 
The main assessment area in PISA 2018 was reading (Mathematics became the main field in 2003 and 2012, and 
science was the main field in 2006 and 2015) (OECD, 2019b). 


Participants 


The data were obtained from the PISA 2018 database which is open access and freely available. Missing values 
were excluded from the analysis. In Turkey 6531 students participated from 189 schools with 3396 being female 
49.3% and 3494 being male 5.7%. In Singapore 6390 students from 166 schools were tested; 3277 were female 
49.1 % and 3399 were male 5.9 %. A large part of the Singapore and Turkey data consisted of grade 10 students 
(respectively 91.3% and 77.8%). 


Instrument and Procedures 


Zylka et al. (2015) introduced ICT engagement scale as a motivation and meta-cognition of ICT literacy. First, 
the ICT engagement scale had a five-dimensional construct, and then it was revised to a four-dimensional construct. 
The revised ICT engagement scale has four factors: interest in ICT, perceived ICT competence, perceived autonomy 
in using ICT and social relatedness in using ICT. ICT engagement scale consists of 4 factors and 21 items in PISA 2018. 

All items were measured by 4-point Likert scale. This scale ranged from 1 for strongly disagree to 4 for strongly 
agree (Zylka et al., 2015). Higher values of 4-point scale show better ICT engagement. The means of the items ranged 
from 2.16 to 3.17 and all were higher than the midpoint of the 4-point scale, 2.0. Descriptive statistics and internal 
consistency of the scale (Croanbach’s alpha) are presented in Table 1. Cronbach's alpha coefficients of four factors 
for each of the two countries ranged from .846 to .876 and were sufficiently high. 

Interest in ICT (IICT): IICT defines an individual’s long-term choice about the tasks, topics, or activities related 
to ICT. Interest in ICT is expected to influence behavior related to ICT and produce positive emotions, learning and 
performance results (Goldhammer et al., 2016). In PISA 2018, this subscale had six items. Internal consistencies of 
interest in ICT were .792 for the Singapore data and .860 for Turkey data. 

Perceived ICT competence (PICT): PICT is the belief of individuals about their knowledge with respect to ICT 
and to use this knowledge (Goldhammer et al., 2016). This subscale consisted of five items. Internal consistencies 
of perceived ICT competence were .806 for the Singapore data and .872 for Turkey data. 

Perceived autonomy in using ICT (PAICT): PAICT projects the individual’s perceived control and self-manage- 
ment in activities related to ICT (Goldhammer et al., 2016). It is expected to experience a sense of control over by 
using ICT tools and to associate their success in using these tools with their own abilities rather than other reasons 
(Kunina-Habenicht & Goldhammer, 2020). This subscale consisted of five items. Internal consistencies of perceived 
autonomy in using ICT were .867 for the Singapore data and .874 for Turkey data. 

Social relatedness in using ICT (SRICT): It expresses the need for the individual to make ICT a subject in com- 
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munication and interaction with others, and also to share their interest, knowledge, experience and activities with 
others (Goldhammer et al., 2016). This subscale consisted of five items. Internal consistencies of social relatedness 
in using ICT were .850 for the Singapore data and .868 for Turkey data. 


Table 1 
Descriptive Statistics and Reliabilities of ICT Engagement Constructs 


Singapore Turkey Overall 
u(SD) a u(SD) a u(SD) a 
ICT 3.14( 0.68) 792 2.82( 0.44) 860 2.98( 0.56) 846 
PICT 2.95( 0.80) 806 2.80( 0.56) 872 2.87( 0.68) 846 
PAICT 3.02( 0.47) 867 2.72( 0.39) 874 2.87( 0.43) .876 
SRICT 2.69( 0.06) 850 2.72( 0.04) 868 2.70( 0.05) 859 


Science Performance: Science literacy defined within the scope of PISA research is considered as the ability of 
students to deal with issues related to science and to reflect on scientific facts. Science literacy requires the ability 
to explain facts, design research methods, and interpret evaluation data and findings scientifically (OECD, 2019a). 
Science performance in PISA measures students’ science literacy in the use of scientific knowledge to determine 
questions, gain new knowledge, describe scientific phenomena, and draw evidence-based conclusions about is- 
sues related to science. The science performances of 79 countries participating in PISA 2018 were evaluated over 
their average scores and results showed that the average science scores were varied between 59 and 336. Turkey, 
ranked 39 in science among 79 countries participating in PISA 2018, it ranked 30th among the 37 OECD countries 
(OECD, 2019d). Singapore was one of the top countries in science. 

Measurement Invariance: Measurement invariance (MI) evaluates whether a construct is measured and 
interpreted in the same way across different groups and is a requirement for making valid comparisons (Putnick 
& Bornstein, 2016). CFA can be used within the scope of SEM to test measurement invariance. There are four hier- 
archical levels of MI in Multi-group CFA and each of these levels is based on the additional equality constraints on 
the model parameters if the previous invariance is met. Configural invariance, which is the first step of assessing 
measurement invariance, explores if the same items measure the same constructs across groups. Factor models 
belonging to all groups to be compared are estimated at the same time. Since this is the baseline model, the valid- 
ity of the configural invariance is evaluated by the model fit measures. The configural model is a baseline model 
to compare with metric invariance (weak factorial invariance). After providing configural invariance, metric invari- 
ance is explored for whether factor loadings are equivalent across groups. Providing invariance of factor loadings 
shows that the construct has the same meaning across groups. After providing metric invariance, scalar invariance 
of item intercepts, is explored for metric invariant items. Scalar invariance (strong factorial invariance) indicates 
mean equivalence in different groups. After providing scalar invariance, the final step is to test residual invariance. 
The residual invariance (strict invariance) is required for full measurement invariance. The residual invariance is not 
required for testing mean differences. Since the residual invariance is insignificant for the interpretation of latent 
mean differences, this step can be skipped in many studies or if this step cannot be provided, latent factor means 
can be compared (Vandenberg & Lance, 2000). 


Data Analysis 


Mplus 6.1 and R Studio tools were used to analyze the data. First step, exploratory factor analysis (EFA) was 
applied to identify the structure of ICT engagement. After demonstrating the convergent validity and the discrimi- 
nant validity, we used the Cronbach's alpha for assessing factor reliability. 

Before investigating invariance, it is important to ensure that the ICT engagement model provides a good fit 
for both countries. Thus, the second step was to test whether the proposed four-factor ICT engagement construct 
fitted the data from each country using the single group CFA approach. In this step, the four-dimensional construct 
of ICT engagement was tried to confirm for both countries. The measurement models of ICT engagement were 
estimated via CFA using Maximum likelihood estimation (ML). Before conducting the CFA, the assumptions of 
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normality were explored. The skewness values of the items varied between (-.92; -.14) and the kurtosis values of 
items varied between (-.76; .94). If the absolute values of skewness and kurtosis are below 1 deviation from nor- 
mality is defined as slight, between 1 and 2.3, moderate, and above 2.3, severe. (Lei & Lomax, 2005). The absolute 
values of ICT engagement items skewness and kurtosis were below 1 the deviation from normality was weak so ML 
estimation was used in this research (Ooi et al., 2012; Simsek & Tekeli, 2015). To evaluate the measurement model 
fit, three most common fit indices (Comparative Fit Index (CFI), Root Mean Square Error of Approximation (RMSEA) 
and Standardized RMR (SRMR)) were used. CFI values more than .90 and RMSEA and SRMR values smaller than .08 
indicate an acceptable model fit, whereas CFI values more than .95 and RMSEA and SRMR values smaller than .05 
suggest good model fit (Hu & Bentler, 1999; Marsh et al., 2005) 

After the ICT engagement construct was provided for each country, in the third step MI was tested using the 
multistage procedure of Multi-Group CFA. A configural model tested for whether the proposed structure of ICT 
engagement would be equal across the two groups. The configural model included the same number of factors and 
the same number of items. There was not equality restriction on measurement and structural parameters across 
countries. The Singapore and Turkey model were estimated at the same time. After the configural invariance was 
provided, the factor coefficients were kept equal for both groups to test metric invariance. The Likelihood-Ratio 
Test (LR) was used to evaluate the model fit in the measurement invariance analysis (Cheung & Rensvold, 2009). 
The change in CFI (ACFI) criterion which is less sensitive to the sample size and alternative fit indices (ARMSEA, 
ASRMR) were also considered. Chen (2007) suggested that such changes should be equal or less than .01 for the 
ACFI, .015 for the ARMSEA and .010 for the ASRMR. Metric invariance held, then, the scalar invariance model was 
examined. Factor loadings and item intercepts were kept equal in both countries. The results showed that scalar 
invariance model fit was acceptable. Thus, both the factor loadings and the item intercepts of ICT engagement 
scale were invariant across Singapore and Turkey. After scalar invariance held, residual invariance was examined. 
The residual invariance was not supported based on the chi-square difference test. As a result, we demonstrated 
scalar invariance of the scale with 4 factors and 21 items across the two countries; thus, we could proceed to struc- 
tural invariance. When we explored the structural invariance, we only tested the factor mean invariance using the 
method that was recommended by Vandenberg and Lance (2000). Singapore was taken as the reference group 
when comparing factor means, so the estimated factor means for countries were actually the differences in fac- 
tor means between groups. We demonstrated strong factorial invariance including configural invariance, metric 
invariance, and scalar invariance of ICT engagement model across groups, so we were able to use ICT engagement 
to meaningful and valid comparisons. 

For the comparison of the effects of ICT engagement factors on performance of science between these two 
countries, we used the Multi-group SEM analysis in the fourth step. Before comparing the path coefficients across 
the countries, each of the two groups were modeled separately (Jiang et al., 2021). After that, the configural SEM 
model was explored considering the two groups simultaneously without any equality constraint on the structural 
path coefficients. 


Research Results 
Exploratory Factor Analysis (EFA) 


First, EFA was performed on a total of 21 items to determine the factors. Kaiser-Meyer-Olkin (KMO), the mea- 
sure of sampling adequacy should be greater than .5, KMO value was .934 which indicated the suitability for factor 
analysis. Bartlett’s test of sphericity was rejected (p< .001). This demonstrated that there were appropriate correla- 
tions in the data for factor analysis. The EFA results using Principal Components Factoring with Promax rotation 
revealed four significant factors. The results showed that 63.50% of the total variance was explained by these four 
factors. Standardized factor loadings of 21 items are illustrated in Table 2. 

As shown in Table 2, all the item loadings were perfect (Tabachnick & Fiddell, 2007). All the items were loaded 
stronger on the related factors than on the other factors and there were no items with noticeable cross-loads. Thus, 
convergent validity and discriminant validity were demonstrated (Churchill, 1979). The internal consistency of the 
factors was evaluated with Cronbach's alpha and they were found to be above the recommended threshold value 
of .60 (Hair et al., 1998). So, the scale was found to be reliable. 
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Table 2 
Standardized Factor Loadings and Cronbach's a 


Factors lICT PICT PAICT SRICT 
658 702 831 793 
770 811 803 820 
Standardized factor 187 680 178 695 
loadings 737 864 837 862 
128 863 831 820 
818 
Cronbach’s o 846 846 876 859 
Single group CFA 


Table 3 shows the model fit indices of the overall data and each of the country for single group CFA. The re- 
sults showed an acceptable fit for Singapore and the overall data. All the CFI values were greater than .90, all the 
RMSEA and SRMR were less than .08. For Turkey, the results showed a good fit. The CFA was .947 and the RMSEA 
and SRMR were less than .05. All the results showed that the four factor of ICT engagement scale was supported 
in both countries. 


Table 3 
Model Fit Indices of the CFA 


df) RMSEA (90%-Cl) CFI SRMR 
Overall 15103.396 (400) .075 (.074-.076) 912 079 
Singapore 6967.367 (183) .076 (.075-.078) 911 063 
Turkey 4081.186 (183) .057 (.056-.059) 947 034 


Multi-group CFA 


Table 4 represents the results of the Multi-group CFA. As shown in Table 4, configural model provided accept- 
able fit for the data (CFI=.92>.90, RMSEA=.053<.08 and SRMR=.062<.10) which indicated the construct equivalence 
of the ICT engagement across Singapore and Turkey. The metric model fit was found to be acceptable (CFI=.92>.90, 
RMSEA=.053<.08 and SRMR=.061<.10) and metric invariance was supported based on the chi-square difference 
test (C’,,=2682.127, Adf=17, p>.001). In addition, ACFl<.01, ARMSEASs.015, and ASRMRs.010 (Chen, 2007). So, 
the metric invariance indicated that the items were equivalently related to the latent factors. Furthermore, the 
scalar invariance model fit was found to be acceptable (CFl=.92>.90 RMSEA=.053<.08 and SRMR= .061<.10). We 
also conducted LR test with the current model (Ax” = 16.840; df = 18; p = 5341)c? ,,=1372.716, Adf=17, p>.001) 
Ax? = 10.273; df = 18; p = .9226).and ACFI, ARMSEA and ASRMR indicated that scalar invariance was held. After 
scalar invariance provided, residual invariance was examined. The tests of differences in fit between residual in- 
variance model and scalar model (¢ =a =194,690; Adf= 21, p<.001); df = 22; p = .000) indicated that residual 
invariance was not established. 
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Table 4 
Fit Indices for Invariance Tests of Measurement Model 


Model CFI RMSEA — SRMR Ay*(Adf) ACFI ARMSEA ASRMR 
Configural Model 91 075 079 - - - 
Metric Model 1 070 068 2682.127(17) 00 -.005 -011 
Scalar Model 92 067 051 1372.716 (17) -.01 -.003 -.008 
Residual Model 90 062 071 3194.690 (21) -.02 000 020 


Even if residual invariance cannot be achieved, groups can still be compared on the latent variable; strong 
factorial invariance is sufficient to make valid comparisons. Singapore was considered the reference group where 
all factor means were constrained to equal 0. Actually, estimated factor means were differences in factor means 
between Turkey and Singapore. The factor means invariance demonstrated that the factor means of interest in 
ICT (-.293, p<.001), perceived ICT competence (-.096, p<.001) and perceived autonomy in using ICT (-.326, p<.001) 
were significantly lower in Turkey than in Singapore while the factor mean (.042, p<.001) of social relatedness in 
using ICT was significantly higher in Turkey than in Singapore. 


Multi-group SEM 


We used Multi-group SEM to explore the equality or invariance of the path coefficients across the two countries. 
Before comparing the path coefficients across the countries, each of the two groups were modeled separately. The 
Singapore model was acceptable (RMSEA=.075, 90% Cl=(.074, .077); SRMR =.064; CFIl=.900). Interest in ICT (.178, 
p<.001) and perceived autonomy in using ICT (.208, p<.001) had positive effect on science performance. Perceived 
ICT competence (-.055, p<.01) and social interaction using ICT (-.305, p<.001) had negative effect on science per- 
formance. The Turkey model also was acceptable (RMSEA=.06, 90% CI= (.059, .062); SRMR=.037; CFI =.937). In the 
Turkey model, interest in ICT (.205, p<.001), perceived ICT competence (.185, p<.001) and perceived autonomy in 
using ICT (.063, p< .01) had significant positive effect on science performance. Social relatedness in using ICT (-.209, 
p<.001) had negative effect on science performance. Singapore and Turkey baseline SEM model fitted data very well. 

To test the invariance of path coefficients, we estimated a configural SEM model for both Singapore and 
Turkey at the same time. The SEM model of the two groups, which was defined as a baseline model, showed an 
acceptable fit for the data (RMSEA =.062, 90% Cl=(.061, .063); SRMR=.067; CFI = .900). 

As the Multi-group SEM was supported, the direct effect of interest in ICT on science performance was restricted 
to be equal across the groups. The Wald test (c?=14.654; df=1; p<.001) showed that the direct effect of interest in 
ICT on science performance was different for Singapore and Turkey. The direct effect of interest in ICT on science 
performance was lower in Singapore (.125, p<.001) than in Turkey (.248, p<.001). After that, the direct effect of 
perceived ICT competence on science performance was constrained to be equal across countries. Wald test (c?= 
37.938; df=1; p<.001) indicated that the direct effect of perceived ICT competence on science performance was 
non invariant for Singapore and Turkey. The direct effect of Perceived ICT competence on science performance 
was lower in Singapore (.055, p<.001) than in Turkey (.092, p<.001). Later, the direct effect of perceived autonomy 
in using ICT on science performance was constrained to be equal across the groups. Wald test (c?= 51.826; df=1; 
p<.001) showed that the direct effect of perceived autonomy in using ICT on science performance was non invari- 
ant for Singapore and Turkey. The direct effect of perceived autonomy in using ICT on science performance was 
lower in Singapore (.088, p<.001) than in Turkey (.151, p<.001). After that, the direct effect of social relatedness in 
using ICT on science performance was set to equal across groups. Wald test (c?= 63.334; df=1; p<.001) showed that 
the direct effect of social relatedness in using ICT on science performance was non invariant for Singapore and 
Turkey. The direct effect of social relatedness in using ICT on science performance was lower in Singapore (-.215, 
p<.001) than in Turkey (-.312, p<.001) 
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Discussion 


The aim of this research was to explore the effects of ICT engagement factors on science performance across 
Singapore and Turkey. The first step was to explore the measurement invariance of the ICT engagement scale 
across these two countries. The single group CFA results confirmed the four-dimensional structure of the ICT 
engagement scale. All items were loaded on the related factors of ICT engagement and all factors were reliable. 
The multi-group CFA results demonstrated that ICT engagement scale with 4 factors and 21 items had strong 
factorial invariance (configural, metric and scalar invariance) across these two countries. The ICT engagement 
had the same construct across Singapore and Turkey, so we were able to use ICT engagement to meaningful 
and valid comparisons. After the strong factorial invariance was confirmed, we tested factor mean invariance by 
treating Singapore as the reference group. The factor means invariance model showed that the factor means of 
interest in ICT, perceived ICT competence and perceived autonomy in using ICT were significantly lower in Tur- 
key than in Singapore while the factor mean of social relatedness in using ICT was significantly higher in Turkey 
than in Singapore. According to the International Telecommunication Union (ITU) 2017 report, Singapore with 
a score of 8.05 had a higher ICT performance compared to Turkey with a score of 6.08. Singapore was at the top 
of this ranking, while Turkey was in the middle. Our results were parallel to levels of ICT development for these 
two countries except social relatedness in using ICT. 

The multi group SEM results demonstrated that interest in ICT, perceived ICT competence and perceived 
autonomy in using ICT had positive direct effect on science performance and these effects were lower in Sin- 
gapore than in Turkey. The social relatedness in using ICT had negative direct effect on science performance 
both in Singapore and in Turkey; this effect was lower in Singapore than in Turkey. The positive and significant 
effects of interest in ICT, perceived ICT competence and perceived autonomy in using ICT indicated that both 
Singapore and Turkey considered the role of ICT in education in a similar manner. These results were consistent 
for both countries, and they were compatible with the ICT policies. The negative effects of social relatedness on 
science performance emerged from the possible adverse side effect of ICT. These results were consistent with 
literature as expected. The higher effect for Turkey might be due to the fact that the students use ICT for social 
purposes rather than learning. 

Students’ interest in ICT showed significant positive effect on science performance both in Singapore and 
in Turkey. This result was also supported by previous studies showing that interest in ICT had a positive relation- 
ship with students’ academic performance (Hu et al., 2018; Jansen et al., 2016; Lee & Wu, 2012; Meng et al., 2019; 
Xiao & Hu, 2019). Some of the studies showed that interest in ICT did not have significant effect on academic 
performance (Juhanak et al., 2019), but most of these studies only analyzed a specific country. Students’ interest 
in ICT was related to the pleasure and positive feelings of using products based to ICT such as mobile devices 
or computers (Zylka et al., 2015). According to the results of this research and other literature, students who are 
more interested in ICT use are likely to have better science learning outcomes. This can be explained as students 
who are more interested in ICT will participate in learning activities using computers or the internet more often 
than other students (Jansen et al., 2016; Lee & Wu, 2012). Also, these students will have a more motivated and 
positive attitude towards science learning with technology (Park & Weng, 2020). Students interested in ICT can 
learn science with an interactive educational software, training software or computer tutorial that are designed 
to help student learning. Students interested in ICT learn when and how they can use different tools and teach- 
ers can also inform students to use the right tools. For example, word processing software helps students for 
organizing ideas, writing homework and projects the works. Students can use spreadsheet for analyzing and 
modeling scientific data. 

Students’ perceived ICT competence also showed a significant positive effect on students’ science perfor- 
mance both Singapore and Turkey. This result supported other studies’ results (Hosein et al., 2010; Hu et al., 2018; 
Selwyn & Husen, 2010). Students with high perceived ICT competence can motivate themselves to solve more 
difficult issues with more effective strategies. According to these findings, students with high perceived ICT 
competence were more likely to use software or online resources to work than those with low ICT competencies 
(Hosein et al., 2010). However, some studies supported that ICT competence had no effect or it had negative 
relationship with performance (Juhanak et al.2016; Xiao & Hu, 2019). The results varied depending on students’ 
grade levels and which country was analyzed (Park & Weng, 2020). The competence in using ICT is considered 
an important skill that students need to acquire in order to be successful in the digital age. 
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Students’ perceived ICT autonomy had a significant positive effect on students’ science performance both 
in Singapore and in Turkey. This result was consistent with the literature (Cardenas-Claros & Oyanedel, 2016; 
Xiao & Hu, 2019; Meng et al., 2019). Autonomy implies that students can organize their learning and use ICT to 
complete tasks and achieve mastery (Fu, 2013). Students with higher autonomy in the use of ICT tend to take 
more control of their learning processes with technology. Moreover, when students realize that they can control 
their learning with ICT, they can strengthen their autonomy in using ICT and learn more by using ICT effectively 
(Serhan, 2009; Park & Weng, 2020). Considering the positive relationship between perceived ICT autonomy and 
students’ science performance, it becomes necessary to organize the education system to support autonomy. 
Both parents and teachers can provide support to meet the autonomy needs of students. Teachers can direct 
students to science group work where they will use tools related to ICT and make them aware of their own 
achievements and shortcomings (Park & Weng, 2020). 

Social relatedness in using ICT was significantly a negative effect on science performance both in Singapore 
and in Turkey, and its effect size was larger than ICT interest, competence, and autonomy. These obtained results 
were similar to some previous studies (Paul et al., 2012; Huang, 2018; Hu et al., 2018). For example, Hu et al. (2018) 
reported that students who used ICT for higher social interaction performed lower. Students who use social 
media too much have less time to learn (Englander et al., 2010). While using ICT for learning purposes, students 
may also lose their focus as they interact with other ICT-related activities such as games, chats, and social media 
alerts. Parents and teachers should alert students to the addiction of entertainment as well as motivate them 
to use ICT for learning purposes (Park & Weng, 2020). 


Conclusions and Implications 


In this research the relationship between ICT engagement factors and science performance was explored by 
Multi-group SEM across Turkey and Singapore based on the data for PISA 2018. These countries were selected for 
comparison purpose because they intended to use ICT in education indicated in their national plans. The success 
of Singapore in science performance was evident. So, it was the reference for comparison to the Turkey which 
gained fair improvement in recent PISA cycles. As a preliminary requirement of Multi-group SEM, the measurement 
invariance of ICT scale across these countries was provided hierarchically validating configural, metric, and scalar 
invariance. The Multi-group SEM result indicated that interest, perceived competence, and autonomy in ICT use 
showed a positive effect while social relatedness in using ICT had a negative effect on students’ science perfor- 
mance both Singapore and Turkey. Although these effects were not different in their directions, their magnitudes 
were higher in Turkey than in Singapore. 

This research contributes to the field of education both because it shows that the ICT engagement construct 
is comparable and because it shows the effect of four factors of ICT on science performance separately between 
these two countries. The results of this research can be a guide in educational policies to be determined to increase 
science performance and it will also be useful in positioning ICT in education system. 

The positive effects of interest, perceived competence, and autonomy in ICT use on science performance can 
be regarded as an opportunity to the developing countries like Turkey considering the benefits of ICT in education. 
Turkey was in the middle rank of science performance according to PISA 2018 results, and it might be starting to 
appear in the positive results of these ICT policies implemented. The negative effect of social relatedness in using 
ICT on science performance should be handled by policy makers. 

This research has a few limitations. First, the responses of students from Singapore and Turkey were analyzed. 
The results of this paper are limited to these selected countries. The research can be extended to different countries 
with different cultures and different educational structures. Second, the relationship between ICT engagement 
factors and science performance was explored. For future studies, the effect of ICT engagement factors on reading 
and mathematics performance can be explored. Third, in this research, only the comparison was made according 
to the countries, and no comparison was made by gender. It is planned to explore the measurement invariance of 
the ICT engagement scale according to gender. 
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Wei Duan wen 2 
theory, this study tested and compared the 
effect size and effect mechanism of formal 
learning experiences (FLE) and informal 
learning experiences (ILE) on 1133 tenth- 
: grade students’ interest in STEM careers 
Introduction 


(ISC) through a paper questionnaire survey. 
The results of structural equation model 
analysis showed that: 1) The total effect of 
ILE on students’ ISC is much greater than 
that of FLE; 2) ILE, STEM self-efficacy (SSE) 
and STEM careers perceptions (SCP) can 
directly affect students’ ISC; FLE and ILE can 
also indirectly affect students’ ISC through 
the mediating role of SSE and SCP. The 
analyses suggest that in order to improve 
students’ ISC, STEM education (espe- 

cially informal STEM education) should be 
strengthened, both formal and informal 
education should pay attention to the 
cultivation of students’ SSE and SCP. 


In today’s knowledge and technological era, science, technology, 
engineering, and mathematics (STEM) workers are the driving force behind 
economic progress of many economies. However, countries around the world 
are facing a shortage of STEM workers to varying degrees. For instance, the US 
Bureau of Labor Statistics (2017) predicted that the demand for STEM work- 
ers would grow by 1 million between 2012 and 2022. Despite the increase 
in demand, the number of students studying in STEM majors is declining 
(National Science Board, 2016). This disparity between demand and supply 
of skilled STEM graduates has led to a shortage of workers in the STEM field. 
The difficulty in filling job vacancies in the STEM field (Wyss et al., 2012) has 
resulted in an effort to improve the quantity and quality of STEM workers 
through reformation of the core goals of the current global education. 

To encourage more students to pursue STEM majors and careers, de- 
velopment of interest in STEM careers is fundamental. Personal interest is 
the critical factor that affects one’s career choice, especially in STEM fields 
(Bahar et al., 2016; Lloyd et al., 2018; Miller, et al., 2018; Nugent et al., 2015). 
Interest in STEM careers refers to personal pursuits and aspirations for STEM 
careers, which can predict students’ inclinations to engage in STEM careers 
in the future to a large extent. However, as adolescents grow, they become 


Keywords: interest in STEM careers, learn- 
ing experiences, social cognitive career 
theory, STEM careers perceptions, STEM 
self-efficacy, structural equation model 


less interested in STEM careers (Chachashvili-Bolotin et al., 2019; Osborne et Ning Wang 
al., 2003), and their aspirations for STEM careers fades. The 2015 Programme Hunan Normal aati pot 
for International Student Assessment (PISA) reported that the proportion of | Nanyang Technological Whiivelsiy, icon 
Chinese 15-year-old students who are willing to pursue STEM-related careers Wu-Rong Xiao 
in the future is relatively low, far below the average level of the OECD (OECD, Hunan Normal sae iis 
2016a). Research (Lindahl, 2007) has shown that the best time to cultivate | Hunan Academy of Education ee oF 
adolescents’ interest in STEM careers is before the age of 13-14. Therefore, Jiong Xiang 
understanding how interest in STEM careers can be cultivated among ado- No.1 High Ni i 
lescents could be one way to enhance the number and quality of individuals Hunan Normal inert Ging 


for the STEM workforce. 
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Earlier studies (Archer et al., 2012; Dewitt 2013; Mohtar et al., 2019) conducted to understand interest 
in STEM careers (ISC) have revealed that factors such as gender, school, society, media, scientific capital and 
personality characteristics exert profound effects on ISC. Comparatively, there are fewer studies relating formal 
learning experiences (FLE) and informal learning experiences (ILE) to students’ ISC (Maiorca et al., 2021; Mohr- 
Schroeder et al., 2014; Roberts et al., 2018). Further, the foci of current studies have not compared the effects 
between FLE and ILE on ISC and centered on university students’ experiences with little attention paid to high 
school students although it has been shown that intention to pursue STEM careers could start in high school 
(Bottia et al., 2015). Consequently, this study was carried out to focus on the effects of FLE and ILE on ISC among 
tenth-grade students, so as to provide theoretical reference for relevant policy makers and educators to cultivate 
students’ ISC. The research questions guiding this study were: 1) Whether FLE and ILE can affect ISC? If so, which 
has greater effect? 2) What is the effect mechanism of FLE and ILE on students’ ISC? 


Literature Review 


Based on Bandura’'s Social Cognitive Theory, the Social Cognitive Career Theory (SCCT) was proposed by Lent 
et al. (1994). This theory integrates external factors such as learning experiences and personal cognitive factors 
(such as self-efficacy, outcome expectation and career interest) dynamically to reveal the process of career choice. 
The SCCT is one of the most powerful theories to explain career choice at present. As such, the literature review 
was based on the three major components of SCCT that have significant effects on ISC: 1) learning experiences, 
2) STEM self-efficacy, and 3) STEM careers perceptions. 


Learning Experiences (LE) 


STEM Learning experiences (whether formal or informal) have effects on students’ ISC. Formal learning 
experiences (FLE) mainly refer to the learning experiences through school organization, classroom teaching 
and other formal learning institutions (Eshach, 2007). FLE are usually regulated by the state, planned with clear 
goals and curriculum knowledge system, and realized by relying on standardized learning materials. FLE in- 
clude professional information provided by teachers, teacher support, teaching methods, courses and teaching 
environment, etc. Rowan-Kenyon et al. (2011) have pointed out that careers information and careers projects 
have significant effects on adolescents’ career planning. STEM curriculum and classroom environment have 
important impacts on students’ interest in STEM (Maltese et al., 2011; Wieselmann et al., 2020). Adding research 
projects and cooperative inquiry activities to STEM courses is conducive to enhancing students’ ISC (Hampden 
et al., 2013; Kang et al., 2017). Teaching strategies (such as student-centered teaching, project-based learning, 
and inquiry-based teaching) can also affect students’ ISC (Lou et al., 2011; Wyss et al., 2012; Zhou et al., 2019). 
As such, with deliberate structures and intentions to engage students with inquiry ways of learning, formal 
education can be designed to affect students’ STEM self-efficacy and STEM careers perceptions, as well as their 
ISC (Mohtar et al., 2019; Wang et al., 2020). 

Compared with FLE, informal learning experiences (ILE) refer to the learning experiences of acquiring new 
knowledge and skills outside the classroom, in informal occasions such as work, life and society. The sphere 
of informal learning includes visiting science and technology museums, other museums and science centers, 
participating in various STEM competitions and summer camps (Eshach, 2007). Numerous studies (Halim et al., 
2018; Kitchen et al., 2018; Melchior et al., 2018; Miller et al., 2018) have shown that informal STEM learning can 
affect students’ ISC. Students who participate in informal STEM-related programs and STEM competitions are more 
interested in STEM careers than those who do not participate, and the effect can be increased by three times if 
students participate in more than one competition (Miller et al., 2018). STEM projects outside the classroom can 
affect students’ choice of STEM-related subjects and college majors (Melchior et al., 2018). Students participating 
in summer STEM programs are 1.4 times more likely to pursue a STEM career than other ones (Kitchen et al., 2018). 
ILE related to STEM also improve students’ STEM self-efficacy and STEM careers perceptions (Halim et al., 2018). 


STEM Self-efficacy (SSE) 


Similar to learning experiences, SSE has an important impact on ISC. Self-efficacy was proposed by Bandura 
(1977) and it refers to a personal subjective judgment or confidence in one’s ability to engage in a certain posi- 
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tion or complete a certain task. Self-efficacy can predict a person’s career interest to a large extent, and students 
are more likely to be interested in careers that they have better abilities and performance, and less likely to be 
attracted to careers that they are not confident in. This is supported by theory-based models, such as SCCT 
(Lent, 1994). Many studies (Halim et al., 2018; Maiorca et al., 2021; Mohtar et al., 2019; Nugent et al., 2015; Wang 
et al., 2020) have demonstrated that SSE is an important factor for individuals to pursue STEM careers. In this 
study, SSE was assessed by students’ confidence in their own abilities and performance in the four subjects of 
STEM (Mohtar et al., 2019). 


STEM Careers Perceptions (SCP) 


Another factor that has effect on ISC is SCP. SCP refer to students’ understanding and knowledge of STEM 
career prospects, skills required and self-development (Franz-Odendaal et al., 2016; Mohtar et al., 2019; Wyss et 
al., 2012). Recent studies have found that SSE and SCP are important factors in predicting whether adolescents 
are interested in STEM careers (Mohtar et al., 2019; Nugent et al., 2015; Schumacher et al., 2009; Wyss et al., 
2012; Mohr-Schroeder et al., 2014; Kitchen et al., 2018). Research by Wyss et al. (2012) has shown that students 
may not be interested in STEM careers if the relevant information about skills, qualifications, requirements and 
employment prospects are not clearly presented to them. As such, the low ISC could be due to the fact that 
most schools do not introduce information about STEM careers to students. Informal STEM learning experi- 
ences can positively influence students’ STEM careers perceptions and enhance their ISC (Kitchen et al., 2018; 
Mohr-Schroeder et al., 2014). 


Research Hypothesis 


SCCT is a theory with the most explanatory power for career choice at present. There are a large number of 
studies (Bahar et al., 2016; Kang et al., 2017; Kier et al., 2014; Maiorca et al., 2021; Mothar et al., 2019; Nugent et 
al., 2015) applying SCCT to study students’ interests in STEM careers and career aspirations. According to SCCT, 
positive environmental factors (such as learning experiences) can improve students’ self-efficacy and outcome 
expectation of careers, deepen students’ perceptions of related careers, and thus enhance students’ interest and 
aspirations for related careers. In this study, the outcome expectations of STEM careers are measured by SCP 
(Mothar et al., 2019). Based on the theoretical constructs of SCCT and the literature reviewed, ISC is affected by 
FLE, ILE, SSE, and SCP, so this study proposed the following four direct hypotheses (H, eH): 

H,: FLE can directly and positively affect students’ ISC. 

H,: ILE can directly and positively affect students’ ISC. 

H,: SSE can directly and positively affect students’ ISC. 

H,: SCP can directly and positively affect students’ ISC. 

In SCCT, positive learning experiences can improve self-efficacy and outcome expectation, and individu- 
als with high self-efficacy also have high outcome expectation for specific careers. Self-efficacy and outcome 
expectation play key roles in the formation of careers interest, both of which are the basis of the development 
of careers interest. In other words, self-efficacy and outcome expectation play important mediating roles in the 
effects of learning experiences on careers interest. Based on the above analysis, this study assumed that SSE and 
SCP play mediating roles between LE and ISC and proposed the following six mediating hypotheses (H,~H,,). 

H,: SSE play a mediating role between FLE and ISC. 

H,: SCP play a mediating role between FLE and ISC. 

H,: SSE and SCP play a chain mediating role between FLE and ISC. 

H,: SSE play a mediating role between ILE and ISC. 

H,: SCP play a mediating role between ILE and ISC. 

H,,: SSE and SCP play a chain mediating role between ILE and ISC. 

In order to explore the effect mechanism of FLE and ILE on ISC, a total of 10 hypotheses were proposed, 
and the hypothesis model (Figure 1) was constructed based on SCCT to test: 1) Whether FLE, ILE, SSE, and SCP 
can affect ISC? 2) Do SSE and SCP play mediating roles between LE and ISC? 
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Figure 1 


Hypothesis Model of the Effect of Learning Experiences on Interest in STEM Careers 
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Research Methodology 


General Background 


Based on the SCCT as the framework, five constructs of this study were FLE, ILE, SSE, SCP and ISC. The final 
questionnaire was obtained through the revision of several well-known international questionnaires related to 
interest in STEM careers (Archer et al., 2013; Buday et al., 2012; Kier et al., 2014; Mohtar et al., 2019; Nugent et al., 
2015) and the test of their reliability and validity. The paper questionnaire survey was conducted among tenth- 
grade students in Hunan Province, China from September to December 2020, and the data were analyzed through 
structural equation modeling to explore the effect mechanism of FLE and ILE on ISC among high school students. 


Sample 


This research selected the tenth-grade students in Hunan Province, China who sat for the new college en- 
trance examination in September 2020. The considerations for the selection of the sample were: 1) Under the 
new college entrance examination format, tenth-grade students have to make choices of courses, subjects and 
examinations, etc. As such, they are in the position to make independent judgments of their career interest. 2) 
Tenth-grade students are relatively mature in physical and mental development, so they are able to accurately assess 
the effects of LE on themselves. 3) Compared with students in Grade 11 and Grade 12, students in Grade 10 have 
less study pressure and have enough time to answer the questionnaire carefully. 4) An early intervention at lower 
grades is more beneficial hence tenth-grade students were chosen. Considering the different levels of economic 
developments in different regions, this study adopted multi-level random sampling method to select tenth-grade 
students in Changsha, Changde, and Xiangxi Autonomous Prefecture of Hunan Province, China to conduct on-site 
questionnaire surveys. Hair et al. (2014) suggested that the sample size should be at least 10 times the number of 
variables. As there were 40 variables in this study, at least 400 samples were needed. In addition, Comrey and Lee 
(1992) considered that a sample size greater than 1000 is the best, so a total of 1240 paper questionnaires were 
collected finally in this study. After inspection, 107 invalid questionnaires (missing answers and random answers) 
were eliminated. In total 1133 valid questionnaires were obtained, and this worked out to an effective rate of 91.37%. 
Among them, 549 were boys, accounting for 48.5%, and 584 were girls, accounting for 51.5%. There were 383 in 
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provincial-level demonstration senior high schools, accounting for 33.8%, 369 in municipal-level demonstration 
senior high schools, accounting for 32.6%, 381 in general senior high schools, accounting for 33.6%. 


Procedures 


In order to protect the privacy of participants, after obtaining the consent of the head teacher and the stu- 
dents, an anonymous questionnaire survey was administered on a class basis. Each class was equipped with one 
or two graduate students majoring in science education as the main tester, who explained the instructions and 
requirements for the participants. The participants filled in the questionnaires within the specified time (about 20 
minutes) and the questionnaires were collected on site after completion. 


Instrument 


The questionnaire was divided into five parts: FLE, ILE, SSE, SCP and ISC (Table 1). All items were measured us- 
ing the 5-point Likert scale, from “strongly disagree” to “strongly agree”. The higher the score, the greater the degree 
of agreement. Among the constructs, the FLE, ILE and SCP parts were adapted from the instrument developed 
by Mohtar et al. (2019). The SSE part was adapted from the instruments developed by Buday et al. (2012), Kier et 
al. (2014) and Nugent et al. (2015). The ISC part was adapted from the instrument developed by ASPIRES Project 
Group, King’s College London, UK (Archer et al. 2013). Combining the suggestions of two experts engaged in sci- 
ence education, one expert engaged in educational statistics, and three front-line teachers, and after two expert 
meetings, an exploratory factor analysis (EFA) was conducted on all items. The EFA showed that the Kaiser-Meyer- 
Olkin (KMO) is .914, and the value of Bartlett’s test of sphericity is 12687 (df=171, p<.001), meeting the judgement 
standards (KMO>.70, p<.05) proposed by Howard (2016). After deleting items with unobvious factor loading, the 
item factor loading of each construct is between .594 and .887, indicating that the structure of the questionnaire 
was well divided. As shown in Table 1, among all the constructs, SSE and SCP include four and three sub-constructs 
respectively, and the score of each sub-construct was calculated by the average score of the items it includes. For 
example, the three sub-constructs of SCP are “Careers Prospects’, “Skills required’, “Self-development”. Each sub- 
construct of SCP contains three items, and the score of each sub-construct was calculated by the average score 
of the three items. 


Table 1 
Constructs of the Questionnaire 


Construct Sub-construct Number of Items Examples of Item 
Demographic variables 3 School, Gender, Grade 

FLE 4 In class, teacher introduce STEM careers to us. 

ILE 4 | have participated in STEM-related competitions. 
Science 4 | can carry out scientific experiments properly. 

eee Technology 4 | can use everyday technological products easily. 
Engineering 4 | can repair a broken toy. 
Mathematics 4 | can draw a graph from the provided data. 
Careers Prospects 3 Those in STEM fields can get jobs easily. 

SCP Skills required 3 STEM careers require creative problem-solving skills. 
Self-development 3 STEM careers contribute to self-development. 

ISC 4 | will pursue a STEM career in the future. 

Total 40 


The reliability and validity of the questionnaire are shown in Table 2. First of all, the overall Cronbach’s a reli- 
ability coefficient of the questionnaire is .916, and the Cronbach's a of each part “FLE”, “ILE” “SSE”, “SCP”, and “ISC” 
are 824, .811,.832, 834, .928 respectively, indicating that the questionnaire has high internal consistency and high 
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reliability. Second, confirmatory factor analysis (CFA) was used to test the validity. The results showed that all the 
standardized factor loadings are between .559 and .935, which reach the standard of 0.5 to 0.95 and the significance 
level, indicating that all the items and constructs are effective. The composite reliability (CR)>0.8 and the average 
variance extracted (AVE)>0.5, indicating each construct has high convergent validity. The correlation coefficients 
between the constructs are less than the square root of AVE in each construct, indicating that the questionnaire 
has good discriminant validity. From the fitting index, RMSEA=.063<.08, SRMR=.043<.08, and all other goodness- 
of-fit indexes are greater than 0.9, indicating that the data fit well. 


Table 2 
Reliability and Validity of the Questionnaire 


Cronbach’s 


Construct Item Factor Loading : CR AVE FLE ILE SSE SCP_ ISC 
FLE1 .760 
FLE2 775 
FLE 824 825 542 736 
FLE3 665 
FLE4 741 
ILE1 754 
ILE2 .728 
ILE 811 817 528 .700 726 
ILE3 721 
ILE4 .702 
Science 833 
Technology 559 
SSE 832 836 567 .708 695 753 
Engineering 697 
Mathematics 880 
Prospects 623 
SCP Skills 853 834 841 643 410 367 479 802 
Development 901 
ISC1 916 
ISC2 935 
ISC 928 929 767 432 548 507 507 876 
ISC3 813 
ISC4 833 
Data Analysis 


Firstly, the analysis of reliability and validity was carried out by the following methods. Cronbach's a was used 
to analyze the reliability of the whole questionnaire and each construct. The content validity of the questionnaire 
was established through consultation with relevant experts engaged in science education and educational statistics. 
The structural validity and convergence validity were tested through EFA and CFA, and the discriminant validity was 
tested by judging whether the correlation coefficient between dimensions was less than the square root of AVE of 
the construct. Secondly, the Harman single factor test (Podsakoff et al., 2003) was performed to ensure that there 
was no serious common method deviation in this study. The structural equation model was constructed, and the 
fitting degree of the model was judged by the goodness of fit indexes. The p-value was used to determine whether 
the direct effects of FLE, ILE, SSE, and SCP on ISC were significant, and the sizes of the effects were analyzed by the 
standardized path coefficients(@). Finally, whether the 95% confidence interval of Bootstrap mediation test (Preacher 
& Hayes, 2004) included 0 was used to test whether the indirect effects of FLE and ILE on ISC were significant and 
the size of the total effect. All the above analyses were performed in SPSS23.0 and AMOS23.0. 
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Research Results 


Harman single factor test was used to test the data for common method deviation (Podsakoff et al., 2003). 
The results showed that there are five factors with eigenvalues greater than 1, which account for 72.12% of the 
variation, and the explanation rate of the maximum factor variance is 40.54%, which is less than the 50% judgment 
standard recommended by Hair et al. (2014). Therefore, there is no serious common method bias in this study. 

The “Structural Equation Model of The Effect of Learning Experiences on Interest in STEM Careers” was con- 
structed in AMOS23.0 (Figure 2), which shows all path coefficients, factor loadings, residual values, etc. of the model 
using the maximum likelihood method. The standardized factor loadings of all observed variables are between 
0.56 and 0.93, which reach the test standard of 0.5 to 0.95. All residual values are positive, indicating that the model 
definition is reasonable. The lower right part of Figure 2 shows the fit of the model. In the structural equation 
model, when the sample size is greater than 200, it is easy to cause the chi-square value of the model to expand, 
resulting in poor model fit (Bollen & Stine, 1993). Therefore, it is recommended to use other goodness-of-fit indexes 
to evaluate the model (Bentler, 1990). Although the x?/df=5.491 of the model is slightly higher than the standard 
value of 5 due to the large sample size, other fitting indicators are at a good level, RMSEA=.063 (<.08, indicating a 
good fit), SRMR=.043 (<.08, indicating good fit), and all other goodness-of-fit indexes are greater than 0.9 (GFI=.930, 
AGFl=.906, NFl=.939, TLI=.939, IFl=.949, CFl=.949). These showed that the constructed model better demonstrates 
the effect mechanism of FLE and ILE on high school students’ ISC, which is suitable for further exploring the rela- 
tionship between variables. Figure 2 also shows the direct standardized path coefficients between the variables. 


Figure 2 
The Structural Equation Model of the Effect of Learning Experiences on Interest in STEM Careers 
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The significance test of the direct effects between each variable is shown in Table 3. The direct effect of FLE 
on ISC is not significant (8=-.061, p>.05), indicating hypothesis H, is not supported; the direct effect of ILE on ISC 
is significant (8=.382, p<.001), indicating hypothesis H, is supported; the direct effect of SSE on ISC is significant 
(B=.127, p<.05), indicating hypothesis H, is supported; the direct effect of SCP on ISCis significant (8=.331, p<.001), 
indicating hypothesis H, is supported. From the perspective of effect size, ILE> SCP> SSE. FLE have no significant 
direct effect on ISC, but they may indirectly affect the ISC among high school students by affecting SSE and SCP. 
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The Direct Effects Test of the Model 


Hypothesis B S.E. t p ah Teenie d 
H,: FLE— ISC -.061 072 -1.212 226 Not supported 
H,; ILE ISC 382 072 7,368 fe Supported 
H,: SSE — ISC 127 .066 2.559 .010** Supported 
H,; SCP — ISC 331 072 9.073 se Supported 


Note: ***p<.001; **p<.01; *p<.05. 


In order to test whether the indirect effects of learning experiences (FLE and ILE) on interest in STEM careers 
are significant, the Bootstrap mediating effect test method (Preacher & Hayes, 2004) was adopted to repeat 
the sampling 5000 times and calculate the 95% confidence interval (Cl). If the 95% Cl does not include 0, the 
mediation effect of the path is significant. Table 4 shows the direct effect, indirect effect, indirect total effect, 
total effect, and the difference between the total effects of FLE and ILE on ISC. The 95% confidence interval of 
the mediating effect (8=.055) of H, (FLE— SSE— ISC) is [.009, .108], which does not contain 0, so H, is supported; 
The 95% confidence interval of the mediating effect (8=.046) of H, (FLE— SCP— ISC) is [.005, .092], which does 
not contain 0, so H, is supported; The 95% confidence interval of the mediating effect (8=.054) of H, (FLE— 
SSE— SCP-— ISC) is [.033,.082], which does not contain 0, so H, is supported. These showed that FLE can affect 
ISC through parallel mediation and chain mediation of SSE and SCP. The indirect total effect of FLE on ISC is 
significant (B=.155, p<.01) and the direct effect of FLE on ISC is not significant (B=-.061, p=.226>.05), so the ef- 
fect of FLE on ISC is completely mediated by SSE and SCP. The 95% confidence interval of the mediating effect 
(B=.049) of H, (ILE— SSE— ISC) is [.008, .091], which does not contain 0, so H, is supported; The 95% confidence 
interval of the mediating effect (8=.003) of H, (ILE— SCP— ISC) is [-.033, .039] which contains 0, so H, is not 
supported; The 95% confidence interval of the mediating effect (6=.048) of H,, (ILE SSE— SCP— ISC) is [.029, 
.074], which does not contain 0, so H,, is supported. The total indirect effect of ILE on ISC is significant (8=.101, 
p<.01), and the direct effect of ILE on ISC is also significant (G=.382, p<.001), therefore, the effect of ILE on ISC is 
partially mediated by SSE and SCP. Overall, the total effect of ILE (G=.483, p<.001) on ISC is much greater than 
that of FLE (B=.155, p<.01), and the difference between their effect size is significant (B=.328, p<.001), indicating 
that the total effect of ILE on ISC is much greater than that of FLE. 


Table 4 
The Indirect Effects Test of the Model 


Direct effect Indirect effect Total effect 
Path Bootstrap 95% C7 
B SE p B SE. B SE p 

Lower — Upper p 

FLE—ISC -.061 072 =.226 

H,: FLE—+SSE—ISC 055 ~=—-.026 009 108 .022* 

H,: FLE—+SCP—ISC 046 = .022 005 092 .029* 

H,: FLE-+-SSE—-SCP—ISC. 054.012 033 082 ae 

Indirect total effect of FLE 155.033 092 221 .001** 
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Direct effect Indirect effect Total effect 
Path Bootstrap 95% C7 
B SE. p B S.E. B SE. p 
Lower — Upper p 
Total effect of FLE 155 033 .001** 
ILE—ISC 382 072 we 
H,: ILE—SSE—ISC 049.021 .008 091 .023* 
H,: LE—SCP—+ISC 003.018 -.033 039 847 
H,,: ILE—+SSE—SCP-ISC 048 = .012 029 074 ane 
Indirect total effect of ILE 101 026 051 154 .001** 
Total effect of ILE 483.049 ace 
Difference 328 ~—-.067 ane 


Note: ***p<.001, **p<.01, *p<.05; Difference="Total effect of ILE” - “Total effect of FLE”. 


Discussion 


Based on the framework of SCCT, this research explored the effect size and effect mechanism of LE (FLE 
and ILE) on high school students’ ISC. The results showed that ILE, SSE and SCP can directly and positively affect 
students’ ISC, and FLE and ILE can also indirectly affect students’ ISC through the mediating roles of SSE and SCP. 

From the results of direct effects, ILE, SSE and SCP significantly affect students’ ISC, which is consistent 
with SCCT and the results of related studies. First of all, this study found that ILE, such as visiting STEM related 
museums, participating in STEM related clubs, competitions and summer camps can promote ISC significantly, 
which is consistent with previous research results (Kitchen et al., 2018; Maiorca et al., 2021; Nugent et al., 2015). 
Secondly, students with higher STEM self-efficacy are more confident that they have the ability to engage in 
STEM careers and achieve good results, and thus more interested in STEM careers (Mohtar et al., 2019; Wang et 
al., 2020). Thirdly, the clearer the knowledge about STEM career prospects, skills required, and self-development, 
the more likely students are to make sober and wise decisions. SCP is an important factor for adolescents to 
be interested in STEM careers (Kitchen et al., 2018; Mohr-Schroeder et al., 2014; Mohtar et al., 2019; Nugent et 
al., 2015; Schumacher et al., 2009; Wyss et al., 2012). However, the direct effect of FLE on ISC is not significant, 
and the study of Zhou et al. (2019) also proved this conclusion that the typical school system fails to improve 
students’ ISC. The reason may be that although teachers are encouraged to conduct inquiry-based teaching, 
problem-based and project-based learning (PBL) during classroom instruction, due to the impact of class hours 
and examination evaluation, teachers paid more attention to the imparting of knowledge in class, rather than 
the cultivation of students’ ISC. 

From the results of indirect effects, LE (FLE and ILE) can indirectly and positively affect ISC through the 
mediating role of SSE and SCP. This is consistent with SCCT, and further expands the application scope of SCCT. 
In other words, SCCT is suitable for research on students’ ISC. Although in many studies (Kitchen et al., 2018; 
Maiorca et al., 2021; Schroeder et al., 2014), LE have important effects on students’ ISC, few studies have found 
and proved the indirect effects of LE. Positive LE (FLE and ILE) can enhance students’ SSE and promote their 
understanding and awareness of STEM careers, which in turn increase their ISC. However, this study also found 
that ILE could not affect ISC through SCP, mainly because the first half of the mediation (the effect of ILE on 
SCP) is not significant. The reason may be that current informal STEM education does not focus on students’ 
understanding and awareness of STEM careers, or it may be that these STEM learning activities are not related to 
real STEM careers. This suggest that STEM activities and programs in schools should have similar characteristics, 
methods and basic principles to the work performed by professionals in STEM field (Kitchen et al., 2018). As 
such, both formal education and informal education, educators should adopt appropriate teaching strategies 
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and teaching activities, cultivate a good teacher-student relationship, stimulate students’ learning initiative, 
optimize students’ STEM learning experiences, so as to more effectively enhance the students’ SSE and SCP, and 
then develop their ISC. 

In terms of the total effect, the effect of ILE on ISC is much greater than that of FLE. This finding is significant 
because few studies have compared the effects of FLE and ILE on ISC, which also provides inspirations for us to 
put forward related strategies. First of all, informal STEM education should be strongly encouraged and promoted, 
especially in the high school stage, because it has been shown that students’ intention to pursue STEM careers 
could start at this stage (Bottia et al., 2015). Thus, educators and policy makers could strengthen collaboration 
between colleges and high schools to develop effective STEM activities that are relevant to real STEM careers; 
The training of STEM teachers (including pre-service and post-service training) should be strengthened, so that 
STEM programs can be carried out successfully; Parents should strongly support their children’s participation in 
STEM-related competitions, clubs, camps, and museums. Secondly, if students who are limited by objective fac- 
tors such as the economic and educational level of the area and the socioeconomic status of the family cannot 
get access to informal STEM education, the following formal education measures should be adopted to enhance 
students’ ISC although informal STEM education has a greater effect on students’ ISC than formal education. This 
study showed that more scientific experiments, collaborative inquiry activities and PBL in classroom instruction 
are conducive to improving students’ ISC, and some studies (Hampden et al., 2013; Kang et al., 2017; Zhou et al., 
2019) also proved this conclusion. Therefore, inquiry teaching and PBL should be adopted in formal education, 
and the competence and literacy of teachers in STEM-related disciplines (mathematics, physics, chemistry, biology, 
geography, technology, etc.) should be improved. In addition, ISC could also be raised in formal education by 
strengthening students’ STEM careers perceptions. However, schools offer limited information on STEM careers 
and textbooks feature limited range of STEM careers, especially in primary school science textbooks. Therefore, 
it is necessary to enrich the career education environment and further explore how to integrate the introduc- 
tions of STEM careers into textbooks. At the same time, the interview videos of STEM professionals could also 
be incorporated and synchronized into corresponding courses. For example, when students are learning DNA, 
they can watch a video of interviews with genetic counselors. 


Conclusions and Future Research 


Referring to the two research questions that “Whether FLE and ILE can affect ISC? If so, which have greater 
effect?” and “What is the effect mechanism of FLE and ILE on students’ ISC?” the results showed that: 1) FLE and 
ILE can positively affect students’ ISC and the total effect of ILE on students’ ISC is much greater than that of 
FLE; 2) ILE, SSE and SCP can directly and positively affect students’ ISC, and LE (FLE and ILE) can indirectly and 
positively affect ISC through the mediating roles of SSE and SCP. Based on the quantitative analysis through 
structural equation modeling, it has been confirmed that ILE play a greater role in promoting students’ ISC, and 
the effect mechanism model of LE (FLE and ILE) on ISC has been established. 

In order to improve students’ ISC, STEM education (especially informal STEM education) should be strength- 
ened. First of all, informal STEM education should be strongly encouraged and promoted. Relevant departments 
should develop effective informal STEM activities, teachers should ensure the implementation of informal STEM 
education, and parents should support children to participate in informal STEM activities. Secondly, in formal 
education, teachers should try to adopt inquiry-based teaching and PBL, enrich the career education environ- 
ment and explore how to integrate STEM career introductions into textbooks and courses. Finally, both formal 
and informal education should pay attention to the cultivation of students’ STEM self-efficacy and STEM careers 
perception in the process. Although this research was carried out in the context of Chinese high school education, 
measures to improve high school students’ interest in STEM careers may have implications for other countries 
as educational issues faced by different countries have some significant similarities, especially the issues of low 
interest in STEM careers among high school students. The conclusions of this study could provide some useful 
references to the teachers and policy makers in STEM, particularly those who are designing STEM learning ex- 
periences. The stakeholders (e.g., students, parents, teachers, policy makers, schools, private sector, etc.) should 
realize the critical role of informal education in fostering students’ interest in STEM careers, and encourage them 
to form a STEM education community, to participate in, support, and develop informal STEM programs. 

Based on the results of the study, future research could consider interviews on top of the questionnaire 
to triangulate the findings. Further, longitudinal research (from primary school to university) could be carried 
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out on the longevity and intensity of adolescents’ interest in STEM careers and how it changes. Previous studies 
have found significant differences in students’ interest in STEM careers in terms of gender, and the formation 
mechanism of interest in STEM careers of different groups is different. As such, in-depth studies can be conducted 
to compare across groups of learners from different genders. Finally, based on different economic and cultural 
backgrounds, how informal STEM programs should be designed to cater to the needs of different student groups. 
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Abstract. The inquiry-based learning mod- 
el can facilitate students’ understanding of 
scientific concepts. Scientific epistemologi- 

cal beliefs (SEBs) are related to students’ 
beliefs about the nature of the process of 
knowledge in science education. However, 
whether the “prediction-observation- 
explanation” (POE) inquiry-based learning 
model can facilitate fifth graders’ concept 
achievement and SEBs in science education 
has not been extensively studied. This study 
selected the unit of Light Refraction to 
explore the effects of POE learning on fifth 
graders’ science concept achievement and 
SEBs. The Light Refraction Test and Scien- 
tific Epistemological Beliefs measurement 
were applied to the two groups prior to and 
following the experiment. The experimental 
group (N=86) participated in POE inquiry- 
based learning, whereas the control group 
(N=88) participated without POE inquiry- 
based learning. The results revealed a sig- 
nificant difference between the two groups, 
with the experimental group learners 
performing better than the control group in 
the concept achievement. In addition, the 
results showed better positive effects of POE 
on experimental group learners’ SEBs in 

the scales of Source and Certainty. Findings 
suggested that learners achieved better 
concept achievements and SEBs with the 
approach of POE inquiry-based learning, 
which pointed to certain implications for 
inquiry-based teaching, as well as in educa- 
tion of future science instructors. 
Keywords: inquiry-based learning model, 
light refraction, prediction-observation- 
explanation, science education, scientific 
epistemological beliefs 
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Introduction 


Cultivating the literacy of science has become the main goal in Chinese 
science education. Ozdem (2010) indicated that scientific concept knowledge 
was one of the important evaluation indicators for scientific literacy, while 
there has been increasing support for inquiry-based learning as a practical 
method to develop science learning and improve learners’ scientific literacy 
(Firman et al., 2019; Suarez et al., 2018). For example, Ozgiir and Yilmaz 
(2017) studied students’ learning of the Acids-bases concepts and showed 
that inquiry-based learning was a usable approach to improve students’ 
achievement. The Cairns and Areepattamannil (2019) study did show some 
positive results for dispositions towards science such as science self-concept; 
however, the results also revealed that inquiry science teaching and science 
achievement has had a significantly negative correlation. Considering the 
inconsistent results, it is necessary to study whether the approach of inquiry- 
based serves science achievement in the different inquiry approaches. Ac- 
cordingly, this study took POE (Prediction-Observation-Explanation) as the 
inquiry science learning model, as it has been proven to be a strongly way 
to learn physical sciences (Latifah et al., 2019). For example, Bunprom et al. 
(2019) showed that the POE learning strategy helped to overcome elev- 
enth graders’ misconceptions of temperature and heat materials. Calik and 
Baycelebi (2020) found that an intervention using the POE learning strategy 
stimulated fifth graders’ conceptions and awareness of healthy nutrition. 
However, a lack of such an intervention study at the level of fifth grader for 
learning the concept of light refraction calls for the current study. Whether 
the POE inquiry learning strategy will also promote the conceptual learning 
of light refraction for fifth graders remains to be seen. In this study, how POE 
affects fifth graders’ concept achievement while they are involved in learning 
light refraction was explored. 
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Mason and Bromme (2010) defined epistemological beliefs (EB) as “individual representations about knowl- 
edge and knowing’ (p. 1). EBs was considered to be a crucial factor in interpreting information and knowledge 
and was therefore related to learners’ conceptual understanding during the learning process (Songer & Linn, 
1991). Peffer and Ramezani (2019) emphasized the importance of exploring the effects of different teaching 
methods on scientific epistemological beliefs (SEBs). In the field of educational research, SEBs have been the 
focus of various science learning studies (Kampa et al., 2016). For example, Liang and Tsai (2010) have stated 
that learners’ SEBs might have an impact on science concept learning, science learning strategy selection (Hsu 
et al., 2014), and other learning behaviors (Lin et al., 2013). Accordingly, increasing attention has been paid to 
the impacts of inquiry-based teaching on learners’ SEBs. For example, Yang et al. (2019) studied the impacts 
of the web-based inquiry learning model on SEBs and found that it was beneficial for eighth grader to predict 
their knowledge integration performance. However, few studies have considered the impacts of applying the 
POE inquiry-based learning model to the SEBs of children, indicating that research on the SEBs of children was 
necessary (Zhou et al., 2019). Thus, how POE can activate or deactivate fifth graders’ SEBs in learning science 
was of interest in this study. 

According to the cognitive theory of multimedia learning (Mayer, 2014), learning is an active process of 
filtering, selecting, organizing, and integrating information based on one’s prior knowledge. Mayer's CTML 
presented the learning model in which the brain interprets learning material via a logical mental construction 
process. In line with this process, this study presented an inquiry-based science learning model, POE, to explore 
the effects on fifth graders’ concept achievements in the Light Refraction unit, and on their SEBs. 


Literature Review 
Inquiry-Based Learning 


The concept of inquiry has a long history in the public school system. Using an inquiry approach to teach 
science has been emphasized (Gillies & Rafter, 2020). Ketpichainarong et al. (2010) stated the inquiry-based 
learning refers to the pedagogical strategies that take scientific inquiry and use general processes as the meth- 
odology of teaching and learning; it emphasizes students asking questions, investigating, and solving problems. 
However, the inquiry has many meanings and interpretations, as Cairns and Areepattamannil (2019) discussed. 
For example, Annisa and Rohaeti (2018) pointed out that inquiry-based learning was an approach that involves 
asking questions, seeking information, and discovering new ideas that were relevant to an event. 

There were many inquiry-based learning models and various inquiry models which had different effects on 
science learning (Hong et al., 2019). Different inquiry-based learning models facilitated students’ understand- 
ing of scientific concepts differently (Rakkapao et al., 2014; Bumbacher et al., 2018). Jerrim et al. (2020) found 
that using inquiry-based teaching was frequently not related to a better science achievement. In other studies, 
students who frequently experienced inquiry-learning approaches in science courses showed lower levels of 
science achievement (Areepattamannil et al., 2020; Firman et al., 2019; Ozgiir & Yilmaz, 2017; Sarwi et al., 2019; 
Suarez et al., 2018). However, using different inquiry-based models yields different learning results. For example, 
the POE inquiry-based model is a potential way to promote students’ acquisition of conceptions (Hong et al., 
2014; Bednar et al., 1992). The study of Arsy et al. (2019) showed that implementing the POE learning strategy 
with the Group Investigation model was beneficial for training students to discover new knowledge, to improve 
the achievement and the quality of their learning. However, there was no particular study focused on using POE 
to explore children’s concept achievement in the area of Light Refraction. 


The POE Inquiry-Based Learning Model 


Pegg (2006) divided the inquiry-based learning model into three categories as follows: a) POCPE model 
involving the phases of prediction, observation, data collection, and explanation; b) POE model involving the 
phases of prediction, observation, and explanation; and c) PCMGE model involving the phases of prediction, data 
collection, measurement, graph making, and explanation. Compared to the other two inquiry-based learning 
models, POE was a more simplified version. In the POE model, the steps are as follows: a) present the students 
with a situation; b) ask them to predict about what will happen when changes are made; c) ask for their reasons 
for their prediction; d) have them perform the change and make observations; and e) have them try to make 
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consistent with any conflict between their predictions and observations (Gunstone, 1990). It was common for 
inquiry-based learning to be arranged into inquiry stages, which were combined to form an inquiry model 
(Pedaste et al., 2015). Anumber of different POE inquiry phases and models have been described; they enabled 
students to connect knew knowledge with their prior knowledge, and helped them absorb the knowledge 
(e.g., Mamun et al., 2020; Hong et al., 2019). Particularly, Costu et al. (2012) showed that the teaching strategy 
of POE-based promoted learners’ achievement. Hong et al. (2014) designed learning activities based on the 
POE inquiry learning model and found that it enhanced students’ interest in the topic and their intention to 
continue to learn science. 

Research studies had found that the POE inquiry learning approach contributed to learners’ conceptual 
understanding and learning science (e.g., Arsy et al., 2019; Ayvaci, 2013; Banawi et al., 2019; Chen et al., 2020; 
Hong et al., 2014; Jasdilla et al., 2018). For example, Chen et al. (2020) implemented the POE strategy in a science 
inquiry study and proved that POE promoted students’ conceptual change and science learning. In addition, 
Jasdilla et al. (2018) tested the POE strategy, and their results showed that it had effects on fifth-grade students’ 
mental models in science learning. Moreover, the study of Ayvaci (2013) implemented POE inquiry learning with 
a group of science teachers and concluded that it was effective and attractive for learning scientific concepts. 
This study focused on how POE could be applied to Light Refraction learning in a science classroom. 


Scientific Epistemological Beliefs 


William Perry was the first to study epistemological beliefs in learning (1970). His study aimed to better 
grasp how students interpreted their educational experiences. Epistemological beliefs refer to beliefs about the 
nature of knowledge and the process of knowing (Hofer & Pintrich, 1997). Hofer and Pintrich (1997) suggested 
epistemological beliefs consisted of four dimensions, where the “certainty of knowledge” dimension and the 
“simplicity of knowledge” dimension address the nature of knowledge, while the “source of knowing” dimension 
and the “justification of knowing” dimension concern the nature of knowing. Hofer and Pintrich’s epistemological 
beliefs theories provided the foundation upon which later research on individuals’ epistemological beliefs has 
been based. For example, based on Hofer’s (2000) works, Conley et al. (2004) suggested that the elementary 
students’ scientific epistemic beliefs should be divided into four dimensions. The four dimensions were “source” 
(e.g., authority or experts are the only source of science knowledge), “certainty” (e.g., there is only one right answer 
regarding science knowledge), “development” (e.g., science knowledge is a subject of constant development and 
change), and “justification” (e.g., evidence from different experiments builds the science knowledge), respectively. 

Many studies have shown that epistemological beliefs promoted scientific learning (Areepattamannil et 
al., 2020). She et al’s (2019) study indicated that epistemological beliefs about science are the most powerful 
predictors of learners’ scientific literacy performance. In Yang et al’s (2019) study, they examined the effects of 
the web-based inquiry learning model on SEBs. However, few studies have explored how POE inquiry learning 
affects children’s SEBs; thus, the present study paid attention to the influence of inquiry-based Light Refraction 
teaching on students’ SEBs. 


Research Questions 


From the results of the several studies described above, there were many benefits to POE. This approach 
which the teacher incorporated was implemented to assist learners in finding their own knowledge and improv- 
ing learning quality. However, none of the previous studies in the POE inquiry learning strategy have tested 
Light Refraction. Moreover, few studies have analyzed the effect on the concept achievements and SEBs of fifth 
graders when implementing POE learning-based teaching. Therefore, the current study explored the effects of 
the POE inquiry-based learning model on fifth graders’ concept achievement in the unit of Light Refraction and 
their SEBs, with the aim of answering the following research questions: 

1. What are the effects of the POE inquiry-based learning model on fifth graders’ concept achievement 
in Light Refraction? 
2. Whatare the effects of the POE inquiry-based learning model on fifth graders’ SEBs in Light Refraction? 
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Research Methodology 
General Background 


In this study, the experiment with pre-test and post-test design was adopted. As shown in Table 1, the control 
and experimental groups were tested by the Light Refraction Test (LRT) and Scientific Epistemological Beliefs 
measurement (SEBs), before and after the experiment. The experimental group participated in POE inquiry-based 
learning, while the control group did not. The study was implemented with fifth graders at a primary school in 
Jiangsu, China in the fall semester of 2020-2021. The same male science teacher, with 15 years of experience 
in science teaching, taught all control and experimental groups, and the groups were provided with identical 
content of Light Refraction. 


Table 1 
Research Design 


Group Pre-test Treatment Post-test 
Control LRT & SEBs POE inquiry model LRT & SEBs 
Experimental LRT & SEBs Conventional LRT & SEBs 
Sample 


The research sample was taken from four classes, with a total of 174 fifth grade students, all aged approxi- 
mately 11 years old. Two classes (46 boys and 40 girls) as the experimental group participated in the POE inquiry 
learning teaching and the other two classes (45 boys and 43 girls) as the control group participated without the 
POE inquiry learning. The questionnaire was issued in November 2020, and a total of 174 questionnaires were 
collected. Randomly filled questionnaires were recorded as invalid. After deleting 23 invalid questionnaires, 151 
questionnaires, including 70 from the experimental group (52.9% boys; 47.1% girls) and 81 from the control group 
(50.6% boys; 49.4% girls), were used for analysis in this study. This study followed the ethical guidelines provided 
by The Nanjing Normal University of China Human Research Ethics Committee. All participants were completely 
voluntary, and expressly informed consent, including the privacy measures necessary to protect their privacy 
interests. In addition, participants were informed that these tests would be used for this research purpose only 
and free to withdraw at any stage. 


Instrument 
The Light Refraction Test (LRT) 


In science education of Chinese fifth graders, the unit of Light Refraction is taught in the fifth grade. It 
is important for their future study that students acquire the concept of Light Refraction properly at that age. 

The Light Refraction Test was used to measure the fifth graders’ concept achievement of Light Refraction 
both before and after the treatment. Yang et al. (2008) suggested that the design of tests should be based on 
the science curriculum and teaching objectives. The test of this study mainly assessed the Light Refraction 
achievement in the basic concepts of Light Refraction. It consisted of five multiple-choice questions and four 
true-or-false items. One item was marked as one point. One sample of a multiple-choice question was: “Which 
of the following does not apply the convex lens principle?” And the choices were presbyopic glasses, magnifying 
glass, and rearview mirror. One sample of a true-or-false item was: “A convex lens can make an object look like 
an inverted image ona screen, whereas a concave lens cannot.’The same questions were applied to the pre-test 
and post-test, but the order differed. A panel of three science teachers determined the face validity and clarity 
of each test item. They also analyzed the relationship between the LRT items and the teaching objectives and 
verified that the LRT instrument had good content validity and measured construct validity, which was suitable 
for the participating students. 
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Scientific Epistemological Beliefs Measurement 


A measurement tool that was frequently utilized to evaluate students’ SEBs was the Conley et al’s (2004) 
questionnaire. This questionnaire contained four subscales and 26 items. The work by Tsai et al. (2011) translated 
Conley et al’s SEBs questionnaire into Chinese and validated this questionnaire by surveying students’ SEBs. This 
study revised this Chinese version questionnaire developed by Tsai and adopted a 5-point Likert scale where 1 
represents strongly disagree and 5 represents strongly agree. After exploratory factor analysis, six items (two from 
“source’, two from “certainty”, one from “development” and one from “justification”) were deleted. The revised SEBs 
measurement’s KMO was .742, and Bartlett p < .05, indicating that it was suitable for the fifth graders. The overall 
alpha value was .88, and the four subscales of SEBs’ alpha values were above .70 (ranged from .76 to .89). The in- 
strument was therefore considered to be reliable enough to evaluate the fifth graders’ SEBs. Detailed descriptions 
of each of the four subscales (20 items) as well as example items are provided below: 

e Source (3 items): Assessing students’ beliefs about external authorities and experts as the only source 
of scientific knowledge. Example item: “In science, you must believe everything you read from the sci- 
ence books” (scored in reverse). 

e Certainty (4 items): Reflected students’ beliefs in the right answer to knowledge in science. Example 
item: “Finding the right answer is the most important part of scientific research” (scored in reverse). 

e¢ Development (6 items): Assessed students’ beliefs about science as a subject that is constantly evolving and 
changing. Example item:“Some scientific ideas today are different from what scientists used to consider.” 

e — Justification (7 items): Reflected students’ beliefs about the role of scientific experiments in science and 
how they justify scientific knowledge. Example item: “Curiosity and thinking about how things work 
lead to ideas about science experiments.” 


Design of a POE Inquiry-Based Learning Model for Light Refraction 


The POE learning model was an approach of promoting interest in science courses (Karamustafaoglu & Mam- 
lok-Naaman, 2015). Pegg (2006) divided the inquiry-based learning model into three categories. In this study, the 
model 2 involving the phases of prediction, observation, and explanation was applied to Light Refraction to help 
the students understand the concepts. In this study, the POE learning involved three rounds. Round 1 provided the 
students with a glass of water and a pen, round 2 with a convex lens and round 3 with a concave lens. Each round 
of the three phases of “prediction, observation and explanation’ are listed as follows and are illustrated in Figure 1. 

Prediction 1: Posed a question to guess (3 minutes). The teacher presented a glass of water and a pencil. 
The question was: After you put a pencil into water, what happens when looking from the top and the side? The 
students predicted the answers based on their prior knowledge and gave their reasons for the prediction. In this 
phase, the students were asked questions as a way of promoting their interest and motivation. They did not know 
whether their answers were right or wrong. 

Observation 1: Explored and observed phenomena (20 minutes). A group of students was given a pen anda 
glass of water; they put the pen into the water, then observed, discussed, and recorded the phenomenon. During 
this stage, the teacher facilitated their investigation by providing them with books, giving them directions, asking 
them questions, and encouraging them to explore and observe. 

Explanation 1: Answered and explained (17 minutes). After exploring and observing phenomena, the stu- 
dents explained the phenomenon of the pencil looking bent after being put into a glass of water. In this stage, 
the students reported their observations to the whole class and to interpret their collective findings. The teacher 
assisted students in finding that traveling from one transparent object to another, light's direction of travel bends 
at the interface. The teacher helped them to understand the phenomenon of a pen in water and the concept of 
Light Refraction. 

Prediction 2: Posed a question to guess (3 minutes). The teacher presented a convex lens. The question was: 
What are the shape and imaging features of a convex lens? The students predicted the answers based on their 
prior knowledge and gave reasons for their prediction. 

Observation 2: Explored and observed phenomena (10 minutes). Every student was provided with a convex 
lens to find reasons for their predictions and performed observations. In this stage, a group of students used a 
convex lens to observe a tree, which required them to collect their own data about convex lenses and the features 
of light passing through a convex lens, and to explore and observe certain phenomena. 


668 


— Wass https://doi.org/10.33225/jbse/21.20.664 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


ISSN 1648-3898 ‘ EXPLORING THE EFFECTS ON FIFTH GRADERS’ CONCEPT ACHIEVEMENT AND SCIENTIFIG 
/Print/ 

EPISTEMOLOGICAL BELIEFS: APPLYING THE PREDICTION-OBSERVATION-EXPLANATION 

ISSN 2538-7 1 38 soniines INQUIRY-BASED LEARNING MODEL IN SCIENCE EDUCATION 


(PP. 664-676) 


Explanation 2: Answered and explained (7 minutes). After exploring and observing phenomena, the students 
explained the shape and imaging characteristics of a convex lens. In this stage, the students reported their observa- 
tions to the whole class and to interpret their collective findings. The teacher assisted the students in finding that 
convex lenses are thick in the middle and thin on both sides, and they can receive light rays from the object and 
concentrate the light to form an inverted enlarged image. In addition, the teacher helped them to link the concept 
of a pen in water and the imaging characteristics of a convex lens. 

Prediction 3: Posed a question to guess (3 minutes). The teacher presented a concave lens. The question was: 
What are the shape and imaging features of concave lenses? The students predicted the answers based on their 
prior knowledge and gave their reasons for the prediction. 

Observation 3: Explored and observed phenomena (10 minutes). Every student was provided with a concave 
lens to find reasons for their predictions and performed observations. In this stage, a group of students used a 
concave lens to observe the class door, which required them to collect their own data about concave lenses and 
the features of light passing through a concave lens, and to explore and observe certain phenomena. 

Explanation 3: Answered and explained (7 minutes). After exploring and observing phenomena, the students 
explained the shape and imaging characteristics of a concave lens. In this stage, the students reported their ob- 
servations to the whole class and to interpret their collective findings. The teacher assisted the students in finding 
that concave lenses are thin in the middle and thick on both sides, and they can receive light rays from the object, 
but cannot concentrate the light to form an image. In addition, the teacher helped them to link the concept of 
imaging features of convex lenses and concave lenses. 


Figure 1 
The POE Inquiry-Based Model (The Experimental Procedure) 
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Procedure 


This study was conducted over four 40-minute lessons. In lesson 1 (before the treatment), both groups of 
students took a pre-test. The LRT and SEBs questionnaires were administered to identify their lens conceptions 
knowledge, and to understand their SEBs, respectively. In lessons 2 to 3 (during the treatment), the control group 
participated in traditional teaching without lenses, at the beginning of which concepts related to Light Refraction 
were explained. The students then read about the content in their textbooks in the classroom, following which 
teacher conducted Light Refraction experiments based on the textbook. In contrast, the experimental group 
students performed experimental activities using the POE inquiry learning model. In lesson 2, the experimental 
group conducted the first round of POE. In lesson 3, the experimental group conducted a second and third round 
of POE. In lesson 4 (after the treatment), both groups took a post-test. The same instruments were used for all 
students to understand the effects of the treatment on their Light Refraction concept achievement and their SEBs. 


Data Analysis 


The pre-test and post-test learning achievement and SEBs data were carried out by SPSS (Statistical Package 
for the Social Sciences). Basic descriptive statistics (Number in a subsample N, mean M, standard deviation SD) of 
the numerical variables were determined. An independent sample t test was used to analyze the difference in the 
pre-test and post-test of the learning achievement, SEBs and four SEBs scales. In addition, the significance level of 
the independent sample t test in this research was set to p = .05. 


Research Results 
Concept Achievement of the Unit of Light Refraction 


Table 2 shows the independent sample t test of the fifth graders learning achievement. No significant differ- 
ence (t = .592, p = .555, effect size d = .101) was found between the two groups in the pre-test. However, in the 
post-test, there was a significant difference (t = -3.835, p < .05, effect size d = -.634) between the two groups. The 
post-test scores of the experimental group (M = 6.89, SD = 1.246) were higher than the control group (M = 6.07, SD 
= 1.340). The inquiry approach of POE used with the experimental group had more success in terms of promoting 
learning achievement compared to the traditional teaching. 


Table 2 
Independent Sample t Test Results of the Two Groups’ Learning Achievement 


Group N M SD t df p d 
Pre- Control 81 4.36 1.248 
2 592 149 555 101 
eS Experimental 70 4.24 1.122 
Control 81 6.07 1.340 
pe -3.835 149 <.05 -634 
ee Experimental 70 6.89 1.246 


Students’ Scientific Epistemological Beliefs 


Table 3 shows the independent sample t test of the fifth graders’ SEBs. As we can see for the two groups, the 
scores in post-test were higher than the pre-test. The pre-test results showed no significant difference between the 
two groups (t= -.334, p =.731, effect size d = -.043). However, in the post-test, there was a significant difference (t= 
-3.676, p < .05, effect size d= -.609) between the two groups. The average SEBs scores of the experimental group (MV 
= 3.47, SD = .294) were higher than the control group (M = 3.28, SD = .329), indicating that the POE teaching which 
the experimental group received had more success in terms of promoting SEBs compared to the traditional teaching. 
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Table 3 
Independent Sample t Test Results of the Two Groups’ SEBs 


Group N M SD t df p d 
Pre- Control 81 3.09 398 
ie -.334 149 731 -.043 
eS Experimental 70 3.11 530 
Control 81 3.28 329 
a -3.676 149 <.05 -609 
Experimental 70 3.47 294 


The independent sample t test was employed to analyze the pre-test of the four SEBs scales. Table 4 shows 
that the four scales of “Source” (t = -1.118, p = .276, effect size d = -.174), “Certainty” (t = 1.56, p = .121, effect size 
d=.252),“Development” (t = -0.175, p = .861, effect size d = -.037) and “Justification” (t = -1.188, p = .237, effect size 
d= -.187) showed no significant difference in the pre-test. 


Table 4 
Independent Sample t Test Results of the Items of the Two Groups’ Pre-test SEBs 


Item Group N M SD t df p d 
Control 81 2.67 1.122 
Source -1.118 144.39 276 -.174 
Experimental 70 2.84 806 
Control 81 2.92 183 
Certainly 1.56 149 121 252 
Experimental 70 2.70 952 
Control 81 3.72 671 
Development -.175 123.946 861 -.037 
Experimental 70 3.75 925 
Control 81 3.91 481 
Justification -1.188 112.6 237 -.187 
Experimental 70 4.03 167 


The independent sample t test was employed to analyze the post-test of the four SEBs scales. In the post-test, 
Table 5 shows that there was a significant difference in the “Source” (t = -5.923, p < .05, effect size d = -.975) and 
“Certainty” (t = -2.914, p < .05, effect size d = -.463) scales between the experimental group and control group. The 
average SEBs score of the experimental group (M = 3.71, SD = .861) was significantly better than that of the con- 
trol group (M = 2.95, SD = .689) in the Source scale. The average SEBs score of the experimental group (M = 3.10, 
SD =.830) was significantly better than that of the control group (M = 2.77, SD =.572) in the Certainty scale. However, 
there was no difference in the “Development” (t = -0.603, p = .548, effect size d = .088) or “Justification” (t = -.829, 
p= 414, effect size d = -.135) scales. 


Table 5 
Independent Sample t Test Results of the Items of the Two Groups’ Post-test SEBs 


Item Group N M SD t df p d 
Control 81 2.95 861 
Source -5.923 149 <.05 -.975 
Experimental 70 3.71 .689 
Control 81 2.77 830 
Certainty -2.914 142.214 <.05 -.463 
Experimental 70 3.10 572 
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Item Group N M SD t of p d 

Control 81 4.10 496 

Development 603 149 548 088 
Experimental 70 4.05 629 
Control 81 4.18 478 

Justification -.829 149 414 -.135 
Experimental 70 4.25 .555 

Discussion 


The Practice of POE Improved Fifth Graders’ Learning Achievement 


Through participating in explorations and discussion activities, the experimental group students regulated 
their thinking about their prior knowledge and reflected on it, in order to form their own connections among 
the science concepts (Jerrim et al., 2020; Zhang, 2019). Accordingly, one of the aims of the study was to explore 
the different influences of the POE learning inquiry model and traditional teaching on fifth graders’ concept 
achievement in the Light Refraction unit. According to the independent sample t test results, students in the two 
groups had similar prior knowledge in Light Refraction, but there were significant differences in their knowledge 
after the intervention, indicating that there were significant differences in the influence of the two methods of 
teaching on fifth graders’ concept achievement of Light Refraction. The result was supported by some previous 
studies, for example, POE inquiry learning promoted students’ learning of scientific concepts (Bunprom et al., 
2019; Fitriani et al., 2020; Karamustafaoglu & Mamlok-Naaman, 2015), indicating that the POE inquiry-based 
learning method was more effective than traditional teaching in terms of learners’ learning of the concepts of 
Light Refraction. 


The POE Inquiry-Based Learning Model Improved Fifth Graders’ SEBs 


The students’ SEBs are generated between knowledge claims and draw on experiences of the knowledge 
building process (Kienhues et al., 2016), where POE regulating the process of knowing is particularly relevant 
for knowledge comprehension (Braten et al., 2015). Based on the independent sample t test results, both the 
experimental and control group learners’ prior SEBs were similar, but there was a significant difference in their 
SEBs after the intervention, indicating that the experimental group could lead to better SEBs than the control 
group. The findings supported the earlier studies which also found positive effects of POE on students’ SEBs 
(Chen, 2017; Cheng, 2018), indicating a significant difference in the “Source” and “Certainty” dimensions for the 
control group and experimental group. However, there was no difference in the “Development” or “Justification” 
dimensions. The results were parallel to the findings of Conley et al.'s (2004), which suggested that students de- 
velop SEBs that more sophisticated “source” and “certainty” were constructed and without significantly changing 
beliefs about “development” and “justifications.” Therefore, the approach of POE inquiry learning facilitated the 
fifth graders’“Source” and “Certainty” dimensions in two 40-minute lessons. 

The POE applied to Light Refraction learning was designed to encourage prediction, observation, and 
explanation, but it ignored argumentation. In addition, the time of the observation phase took half of each les- 
son. It was obvious that emphasizing observation limited the opportunities of the learners to use evidence to 
debate their ideas and to reflect on their inquiry activities. This might account for the lack of significant change 
in the dimensions of development and justification. The underlying mechanism of this change should be more 
validated in studies that make comparisons with science classrooms that adopt inquiry-based teaching strate- 
gies with a focus on argumentation and reflection (Herrenkohl et al., 1999). 


Conclusions and Implications 


This study presented the three round POE inquiry-based learning in the Light Refraction unit and explored 
its effects on fifth graders’ concept achievements and their SEBs. It could be concluded that learners achieved 


672 


—  Wsss7 https://doi.org/10.33225/jbse/21.20.664 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


ISSN 1648-3898 . EXPLORING THE EFFECTS ON FIFTH GRADERS’ CONCEPT ACHIEVEMENT AND SCIENTIFIC 
/Print/ 

EPISTEMOLOGIGCAL BELIEFS: APPLYING THE PREDICTION-OBSERVATION-EXPLANATION 

ISSN 2538-7 138 joniines INQUIRY-BASED LEARNING MODEL IN SCIENCE EDUCATION 


(PP. 664-676) 


better concept achievements and SEBs in POE inquiry-based learning. In addition, it showed better positive 
effects of POE inquiry-based learning on learners’ SEBs in the subscales of “source” and “certainty”. Therefore, 
when using the POE inquiry-based model teaching, instructors were suggested to strengthen the cultivation 
of learners’ SEBs in the scales of “development” and “justifications.” 

The implementation of the POE inquiry-based learning model is expected to provide theoretically and 
practically help for learners to learn simple science concepts. Theoretically, this implementation can help learn- 
ers to improve their concept achievement, and to build views of their own knowledge and knowing in science. 
Practically, the overall results of the present research suggest that when learners practice POE, it can help them 
understand Light Refraction and achieve better learning performance. Therefore, the POE model can be applied 
to promote students’ science learning and help learners gain a better understanding of exactly the generation 
and development of scientific knowledge. This study has some enlightenment for instructors that it needs to 
design different inquiry models according to different scientific concepts so as to effectively improve learner's 
concept achievement. It is of great significance to the education system of science instructors and enhancing 
their competencies for inquiry-based teaching. 

The implementation of the POE inquiry-based learning model is expected to provide the help for improving 
learners’ SEBs. Learners’ SEBs can be strengthened based on applying the POE learning-based model to teach- 
ing. Therefore, it is suggested that instructors use the POE learning-based model in science courses to improve 
learners’ SEBs. It is very important that the instructor applies inquiry teaching strategies to make learners play 
an active role in the teaching process. This study has an important contribution for instructors to design the 
teaching process using POE inquiry-based model teaching strategies. In scientific learning inquiry, there is often 
more than one prediction or question. If all of the predictions and questions are put at the beginning phase, it 
may be difficult to focus on the one prediction. However, if one POE model is explored one prediction in one 
round, many details can be focused on. Therefore, if there are a lot of predictions, it is suggested that instructors 
construct learners’ SEBs one prediction at a time, and one POE model at a time. 


Limitations 


This study has some limitations. The study implemented one unit of teaching. Therefore, the suitability of 
the POE inquiry model for other new units or new subjects is not certain. Thus, future studies can select more 
units or topics of science to conduct POE for students to learn and examine their SEBs and learning achievement. 
Teachers have been found to hold sophisticated scientific epistemological beliefs. Education can be enhanced if 
teachers are asked to examine their belief structures. This might in turn impact their SEBs beliefs about teachers’ 
and students’ roles. Therefore, in the future, it is suggested that the focus be on research of teachers’ epistemo- 
logical beliefs, especially in science education, and on what types of experiences impact the development of 
these beliefs and conceptions. In addition, due to the period of this study being short, with just 80 minutes over 
a period of two classes, the results of this study suggested that learners developed SEBs that the more sophisti- 
cated “source” and “certainty” were constructed and without significantly changing beliefs about “development” 
and “justifications.” Therefore, future studies may implement longer experiments to explore if the POE inquiry 
learning teaching can facilitate the students’“Source” and “Certainty” dimensions. 
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Introduction 


Nowadays, the developments of information and communication 
technology occurring with globalization have affected the lives and econo- 
mies of all nations. The common motivation for countries around the world 
today is to improve the educational system to make the new generation 
more innovative and creative (Lederman, 2008). Demand has also increased 
for countries which would develop new technologies and produce more 
valuable and innovative products (Ergun, 2019). For example, workers in 
the field of science, technology, engineering, and math (STEM) are favored 
in the United States (Carnevale et al., 2011; Rothwell, 2013), and it has been 
predicted that the demand of STEM workers will continue for years (National 
Academy of Engineering, 2008; Unfried et al., 2015). It is stated that STEM 
skills are necessary for everyone, not only those who want to pursue STEM 
careers, but also those who do not (Yerdelen et al., 2016). Students need 
to actively improve their STEM capabilities to cope with the challenges 
of globalization and the development of the knowledge-based economy 
(Kristen et al., 2012). Therefore, the economic globalization encourages 
individuals to make adequate efforts and preparations to improve STEM 
literacy and to pursue STEM related careers (Yerdelen et al., 2016). 

In educational field, with countries recognizing the role of STEM 
education in the future’s economy, improving students’ STEM learning has 
become a top priority (McMahon & Showers, 2012). STEM education has 
been considered as a government policy in the United States, (National 
Research Council [NRC], 2010) and the interest in STEM disciplines has 
increased in many European countries (Corlu et al., 2014). STEM Education 
has also been regarded as one of the hot topics in the ministry of education 
of many countries, such as China (Zhou et al., 2019) and Turkey (Yerdelen 
et al., 2016). Generally, improving students’ attitudes towards STEM could 
raise the possibilities of students to choose STEM careers in the future. 
Therefore, the importance of STEM education requires more attention to the 
attitudes towards STEM-related domains from kindergarten kids through 


https://doi.org/10.33225/jbse/21.20.677 


JOURNAL 
OF®BALTIC 
SCIEN CE 
EDUCATION 


ISSN 1648-3898 reins 
ISSN 2538-71 38 joni 


Abstract. Most studies have concentrated 
in assessing students’ overall attitudes 
towards science, mathematics, and engi- 
neering/technology or the attitude towards 
individual STEM domain. The present 
research aims to explore primary students’ 
gender and grade differences of their STEM 
domain-specific attitudes including self-ef- 
ficacy and expectancy-value beliefs, as well 
as their correlations. The results showed 

no detected significant effects among 
these different STEM domains in the overall 
attitudes, the overall self-efficacy beliefs, 
and the overall expectancy-value beliefs for 
primary students. The correlations between 
self-efficacy and expectancy-value were 
much stronger for the science domain and 
engineering/technology domain than the 
mathematics domain. No gender difference 
of the self-efficacy beliefs was detected 
except in the mathematics domain, and 
the result that lower primary students 
performed significantly better than upper 
primary students in the self-efficacy was 
also mainly contributed by the grade differ- 
ence in the mathematics domain. Whereas 
no different expectancy-value beliefs 
existed across genders and grade levels in 
various STEM domains. The present results 
reported some unique performances by the 
primary school students compared to the 
elder group. 

Keywords: expectancy-value, gender 
differences, grade levels, self-efficacy, STEM 
attitudes 
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K-12 school students (Business-Higher Education Forum, 2010; President’s Committee of Advisors on Science 
and Technology [PCAST], 2010). 

However, many countries around the world are facing a common challenge of decreasing numbers of students 
who show interest in the STEM-related disciplines (National Science Board [NSB], 2012; Osborne & Dillon, 2008). It 
is reported that more than three times as many graduates in humanities and social sciences than those in STEM- 
related major in the Organization for Economic Co-operation and Development (OECD) countries (OECD, 2015). 
It is noted that only a small number of students starting higher education have chosen STEM-related disciplines 
among the OECD countries (OECD, 2017). Additionally, compared with other fields, more students drop out of 
STEM disciplines which they have chosen in the first place (Reinhold et al., 2018). It is stated that the number of 
students in higher education institutions choosing STEM related majors is much lower than expected (Shapiro & 
Sax, 2011). As it has been reported by University of California at Los Angeles (UCLA), many K-12 and post-secondary 
students lost interest in STEM, with nearly half of engineering and science majors switched to other majors or failed 
to get a degree (Drew, 2011). Besides, according to previous research, the number of students who value STEM in 
universities in developing countries has also decreased over the years (Akgunduz, 2016). 

Naturally, poor academic performance affects the number of students who complete higher education 
degrees and enter STEM-related professions (Christensen et al., 2014). Many capable students do not choose 
STEM-related fields as their careers in many countries such as United States. As a result, the rate of choosing 
STEM-related fields as a career is generally low (Kizilay, 2018). Globally, the demands about labor force in STEM- 
related fields are not being adequately met (Atkinson, 2013; Moakler & Kim, 2014). For example, only 300,000 
students with STEM-related majors in the United States are graduating each year, while the demand for the 
workforce is expected to be close to one million (Holdren & Lander, 2012). 


Literature Review 


Due to the continuous extension of the research on students’ attitudes, researchers’ understanding of at- 
titudes has been deepened, which leads to the continuous expansion and evolution of the definition of attitudes 
(Luo et al., 2019). Students’ attitude towards a subject is considered to be an important correlation of achievement 
motivation, which consists of two important components, namely, self-efficacy belief and expectancy-value 
belief (Eccles & Wigfield, 2002). Self-efficacy was described by Bandura (1997) as an individual’s confidence in 
their ability to accomplish a task. Researchers have found increasing evidence that self-efficacy could be con- 
sidered as a predictor of academic achievement (Multon et al., 1991; Zimmerman & Bandura, 1994). According 
to the previous studies, students are more likely to continue postsecondary education in STEM domain if they 
show higher self-efficacy in math (Wang, 2013) or science (Scott & Mallinckrodt, 2005). As for expectancy-value 
theory, first proposed by Atkinson in the 1950s, it has since been expanded into educational field to address 
students’ achievement-related choices (Wigfield & Eccles 2000). Expectancy-value theory holds that individuals 
periodically evaluate their likelihood of achieving certain goals and evaluate the specific value gained or lost 
in achieving those goals (Eccles & Wigfield, 2002; Wigfield & Eccles, 2000). According to the different connota- 
tions of expectancy-value belief, Bandura divided it into two sub-dimensions, that is, outcome expectations 
belief and efficacy expectations belief (Luo et al., 2019). Outcome expectations belief represents that certain 
behaviors will lead to certain outcomes (e.g., “the belief that practicing will improve one’s performance”) and 
efficacy expectations belief indicates whether individual is capable of performing effective behaviors necessary 
to generate outcome (e.g., “Il can practice sufficiently hard to win the next tennis match”). Whether one has high 
expectations has been demonstrated to be closely related to one’s adherence to advanced math and science 
courses (Fan, 2011; Simpkins et al., 2006). When studying career aspirations model, expectancy-value theory is 
regarded as a supplement to self-efficacy theory (Schunk, 1991; Wigfield & Eccles, 2000). For the purposes of 
the present research, students’ attitudes are defined in alignment with Eccles and Wigfield’s (2002) description 
focusing on self-efficacy and expectancy-value beliefs. In addition, this research takes interest in such beliefs 
towards STEM disciplines across genders and grade levels. 


Research on the attitudes towards various core STEM subjects 


Since the 1970s, research on students’ attitudes towards science and math has produced an impressive 
body of research literature (e.g., Aiken, 1970; Gardner, 1975; Osborn et al., 2003; Potvin & Hasni, 2014b; Simpson 
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& Oliver, 1990). The research literature has also emerged with surveys measuring students’ attitudes towards a 
single STEM discipline (Unfried et al., 2014). Typically, students’ interests and expectations in math or science 
are of primary concern to the researchers (Wiebe et al., 2018). For instance, the Test of Science-Related Attitudes 
(TOSRA) is an instrument that consists of seven subscales with 10 items each, measuring student attitudes towards 
science in the secondary education period (Fraser, 1978). Affective Elements of Science Learning Questionnaire 
is another assessment of student attitudes towards science (Williams et al., 2011). Another instrument is the 
Attitudes Towards Mathematics Survey (Miller et al, 1996), which aims at measuring student attitudes towards 
mathematics. In contrast to the literature that measures students’ attitudes towards science and mathematics, 
less research has focused on students’ attitudes towards technology and engineering areas (Johnpaul et al., 
2018). Different tendencies of this kind of research have even led a part of researchers to a question whether 
science and mathematics alone are worthy of attention, rather than integrating technology and engineering as 
elements to be the whole of STEM area (Lederman & Lederman, 2013). Studies which focus on technology aspect 
of STEM, tend to view the technology itself as a toolbox of skills that could be applied to science and mathematics, 
rather than as a primary and unique domain of STEM (Kennedy et al., 2018). Unlike other instruments, Erkut and 
Marx (2005) developed an instrument for the survey of attitudes towards multiple STEM areas: science, math, 
and engineering/technology. This instrument could be applied to assess students’ attitudes towards different 
STEM disciplines for understanding their similarities and differences (Johnpaul et al., 2018). It is necessary to 
systematically gather data of attitudes for young students across various STEM domains (Minner et al., 2012). 

In the present research, the differences were examined, in attitudes towards all core STEM subjects (science, 
math and engineering/technology) and interests in STEM careers among 1st through 6th grade students. This 
research also focused on primary students’ performances in self-efficacy and expectancy-value beliefs towards 
various STEM domains. 


Student attitudes towards STEM across genders and grade levels 


In terms of gender, it was found that contradictory evidence existed on student attitudes towards STEM 
(Toma & Greca, 2018). Most research studies suggested that boy students tended to have more positive atti- 
tudes towards STEM than girl students (Greenfield, 1996; Jarvis & Pell, 2005; Jones et al., 2000). On the contrary, 
some studies indicated that students did not exhibit significant difference in the attitudes towards STEM across 
genders (Akpinar et al., 2009; Zhou et al., 2019). Even, it has also been found that the attitude of boy students 
towards STEM was less positive compared to girl students (Boone, 1997). As well, other studies have examined 
gender differences in student attitudes towards STEM sub-disciplines (Wiebe et al., 2018), such as science (Ak- 
pinar et al., 2009; Toma & Greca, 2018), mathematics (Watt et al., 2012), engineering, and technology (Unfried et 
al., 2014). Eccles pointed out that the underlying reasons for the differences in STEM attitudes based on gender 
were nuanced (Eccles & Wigfield, 2002). 

More specifically, recent reviews which covered gender differences across STEM disciplines found that 
gender differences in self-efficacy existed across most of the STEM domains (Eddy & Brownell, 2016; Cheryan et 
al., 2017). Cheryan et al. (2017) stated that gender differences in self-efficacy predicted the different participa- 
tion rates for girls in STEM disciplines. Compared with girls, boys have higher expectations for success in STEM 
fields (Yee & Eccles, 1988). This difference could explain the decreased enrollment in STEM-related courses for 
girls (Watt et al., 2012). Gender differences in STEM achievement may be influenced by the amount of value that 
boys and girls place on STEM domains as well. However, many studies indicated no gender differences in the 
expectancy-value levels of STEM (Eccles, 2009). It can be seen that gender differences in students’ attitude states 
in high school or college have been examined individually in prior studies, but not students’ self-efficacy and 
expectancy-value beliefs in primary school. It is necessary to do further investigation among primary graders. 

From the aspect of grade levels, a number of studies have paid attention to the grade differences in STEM 
attitudes involving students from primary school through university. Many studies have determined a common 
result that older graders have a less positive attitude towards STEM domains compared to younger graders 
(Potvin & Hasni, 2014a; Unfried et al., 2014). For instance, a group of international and Australian longitudinal 
studies showed that student attitudes towards STEM steadily declined due to students’ transitioning to early 
high school (e.g., Speering & Rennie, 1996). Attitudes towards science have also been found to decline from 
upper primary school to middle school in a longitudinal study (Unfried et al., 2014). The previous research has 
determined that lower primary students performed much more positive attitudes than upper primary students 
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(Zhou et al., 2019). However, there were contradictory results in some existing studies, in which some showed 
no significant differences across grade levels, while others supported an increase in student attitudes towards 
STEM domains as the grade grew (Akpinar et al., 2009; Ali et al., 2013; Said et al., 2016). In addition, grade level 
has been considered to be a significant factor, having interactions with gender on STEM learning attitudes 
(Wiebe et al., 2018). In general, student attitudes towards STEM fields have been examined across grade levels 
for different graders in prior research. The prior research on primary students’ attitudes towards STEM also con- 
tributed to the related literature (Zhou et al., 2019). A more nuanced research on differences of attitudes across 
various STEM disciplines is worthy of subsequent exploration. Further, it makes sense to investigate the grade 
level difference of students’ self-efficacy and expectancy-value beliefs towards STEM domains in primary grades. 


The relationship between self-efficacy belief and expectancy-value belief 


There are various correlations among various variables of STEM attitudes in the literature. One research, 
on the one hand, found little relation between self-efficacy and outcome expectancy in STEM-related domains 
(Maddux et al., 1986). On the other hand, for the science attitudes, moderate correlations exist between the 
perceived usefulness of school science for scientific careers and each of self-efficacy, enjoyability, relevance, 
and intentions. For the mathematics attitudes, there is a moderate correlation between self-efficacy and enjoy- 
ability. As well, there are moderate correlations between the different attitudes towards design technologies 
(Johnpaul et al., 2018). 

Regarding two essential components of student attitudes towards STEM, Bandura’s self-efficacy theory 
focuses on expectancy for success based on expectancy-value theory and attribution theory (Luo et al., 2019). 
Manning and Wright (1983) found a strong correlation (r=.75) between self-efficacy and outcome expectancy. 
Self-efficacy was modeled as a determinant of outcome expectancies, interests, and intentions (Fouad & Smith, 
1996). Self-efficacy is considered as a compelling predictor of STEM academic success and the willingness of 
students to set challenging goals like a STEM educational pathway. Wang (2013) found that a college student's 
intent to major in STEM was directly affected by his or her 12th-grade math achievement, exposure to math and 
science courses, and math self-efficacy beliefs. During high school, having high expectancy-value beliefs has been 
found to be associated with a student's persistence in taking both advanced science and mathematics courses 
(Simpkins et al., 2006). Research studies have shown that students were more likely to pursue postsecondary 
schooling in STEM fields if they had success in mathematics (Wang, 2012) or high self-efficacy in science (Scott 
& Mallinckrodt, 2005) in earlier grades. 


Research Questions 


In the early studies of the development of social psychology, researchers explored the impact of demograph- 
ics including age, gender, and race on self-efficacy, outcome expectations, and career interests (Fouad & Smith, 
1996). In one of the previous studies, the researchers studied the factors affecting students’ attitudes towards 
STEM domains and found that gender had an interactive effect on attitudes and interests in STEM professions 
(Unfried et al., 2014). Recently, some studies have found a positive effect of STEM oriented interventions on 
students’ STEM career interests (e.g., Peterman et al., 2016; Xie & Reider, 2014). The researchers have emphasized 
that self-efficacy and expectancy-value form a core structure that can influence motivation and persistence in the 
academic trajectory (Schunk, 1991; Wigfield & Eccles, 2000). What's more, relevant studies have demonstrated 
the need to distinguish differences between general self-efficacy and specific self-efficacy towards specific 
academic domains (Chen et al., 2000). However, there has been less related research about STEM attitudes 
involving primary students from grade one to grade three as emphasized in the previous research (Zhou et al., 
2019). It suggests that there is a need to study the attitudes towards STEM among students through all grade 
levels in primary school. Therefore, the purpose of this research was to extend the previous research (Zhou et 
al., 2019) to examine differences of primary students’ domain-specific STEM attitudes in terms of genders or 
grade levels, and to explore the correlation between students’ STEM self-efficacy and expectancy-value beliefs. 
Three research questions were as follows: 

1. What is the correlation between STEM domain-specific self-efficacy and expectancy-value beliefs? 
2. Is there a significant difference of primary students’ STEM domain-specific attitudes in terms of 
gender or grade level? 
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Research Methodology 
General Background 


The present research was an extension of the previous research (Zhou et al., 2019), which explored the 
performance of primary students in overall STEM attitude and the improvement of students’ STEM attitude 
after receiving a twelve-week integrated STEM project. In this previous research, STEM attitude was regarded 
as a unified disciplinary attitude, which was not divided neither according to the different domains of STEM 
nor according to the dimensions of self-efficacy belief and expectancy-value belief. The present research also 
took primary school students as the research object but focused on understanding more multi-layered states 
of students’ STEM attitudes. The data for this research were excavated from the same research program in the 
previous research (Zhou et al., 2019), in which the data were collected in 2018, and the in-depth data mining 
was conducted during the period 2019-2020. From the perspective of different domains, it is worth exploring 
whether there were differences in students’ attitudes towards STEM in various STEM domains and extracting 
gender differences and grade level differences on this basis. In addition, from different dimensions of STEM 
attitude, this research also aimed to explore whether there were differences in the self-efficacy belief and 
expectancy-value belief of primary school students, and whether the difference was found in different STEM 
fields, different genders, and different ages. 


Sample Selection 


In order to study how primary students exhibit different attitudes towards STEM according to gender and 
grade level, and to explore the relationship between primary students’ STEM self-efficacy and expectancy-value, 
the participants were selected from grade one through grade six in primary school in China. The primary school 
is an ordinary one in a Guangdong Province in China, with no special teaching intervention in STEM educa- 
tion beyond the curriculum standards of the Ministry of Education (MoE) of China. All the students voluntarily 
participated in the present research and with the consent of their parents. Based on the above principles, the 
participants and the ratio of boys to girls in different grades were different. The population contained 127 boys 
and 54 girls, with a total number of 181 primary school students. The distribution of boys and girls in different 
grades is shown in Table 1. 


Table 1 
The Distribution of Boys and Girls in Different Grades in the Research 


Grade 1 Grade 2 Grade 3 Grade 4 Grade 5 Grade 6 Total 
Boys 7 17 34 36 11 22 127 
Girls 5 4 14 15 10 6 54 
Total 12 21 48 51 21 28 181 


Instrument and Procedures 


There have been many studies designed to provide validity evidence for evaluating students’ attitudes 
towards a single discipline or integrated STEM areas at different levels of the education system (e.g., Adams et 
al., 2006; Lent et al., 2008). Although there were a limited number of studies attempting to investigate students’ 
STEM attitudes during the K-12 school year, some new assessments still show promise (e.g., Unfried et al., 2015). 

In the previous research, the S-STEM survey which developed from North Carolina State University was 
used to assess primary students’ STEM attitudes (Unfried et al., 2015; Zhou et al., 2019). Within the entire S-STEM 
survey, there are 26 items assessing students’ STEM attitudes based on three dimensions: science (9 items), 
math (8 items), and engineering and technology (9 items) (Yerdelen et al., 2016). The survey also contains 11 
items assessing students’ 21* century skills. As this subscale was not relevant to the present research, it was not 
introduced and analyzed below. Each dimension consists of both self-efficacy items, such as ‘Il am sure | could 
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do advanced work in math’, and expectancy-value items such as‘I will need a good understanding of math for 
my future work’ (Wiebe et al., 2018). Table 2 describes items representing the attitudes towards different disci- 
plines and the statements comprising categories. The S-STEM survey uses a five-point Likert scale from strongly 
disagree to strongly agree ona scale of 1 to 5. 


Table 2 
Descriptions and Categories of Items in S-STEM Survey 


Discipline categories Statements comprising categories 
Self-efficacy Expectancy-value 
Math 1,3,4,5,7,8 2,6 
Science 9,14,16 10,11,12,13,15,17 
Engineering and Technology 18,20,21,23,26 19,22,24,25 


The S-STEM survey has an upper primary version for 4th and 5th graders in primary schools as described 
in the previous research (Wiebe et al., 2018). The validity evidence on the S-STEM survey has been proved by 
previous research studies (Unfried et al., 2015; Luo et al., 2019). In the present research, this version has been 
translated into Chinese as simple and suitable, as possible for lower primary graders by two of the researchers. 
It took each student about 20 minutes to complete the S-STEM survey. Especially for first and second graders, 
the teacher needed to read each item when necessary to make sure the meanings of all items were understood 
by students (Zhou et al., 2019). 


Data Analysis 


SPSS Statistics was applied for the test on S-STEM to establish their reliability evidence. The reliability 
(Cronbach's alpha) coefficient was .896, indicating sufficient consistencies in the outcome of the test in present 
research. The data support the use of the S-STEM survey for the investigation. Besides, various categories of 
student attitudes towards STEM across genders and grades (lower primary students and upper primary students) 
are compared, using the analysis of variance (ANOVA) statistic and the independent sample t-test statistic. Also, 
the correlation analysis and linear regression analysis were used to test the correlation between primary students’ 
self-efficacy and expectancy-value beliefs towards STEM subjects. 


Research Results 
Students’ STEM Domain-specific Self-efficacy and Expectancy-value Beliefs 


The overall mean score of the whole population in the STEM survey was higher than 3 (M=3.61, SD=.81), 
indicating a positive exhibition of students’ STEM attitudes. One of the research aims was to explore whether 
primary students exhibited different attitudes towards three sub-dimensions as science, mathematics, and 
engineering/technology. Therefore, this research compared students’ performances on the overall STEM atti- 
tudes and different categories of attitudes including self-efficacy and expectancy-value, across three different 
sub-dimensions of STEM areas. 
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Table 3 


Correlation Matrix for Attitudinal Constructs within Students’ STEM Domain-specific Attitudes Profile 


Correlation coefficient 


N STEM attitudes Self-efficacy Expectancy-value hatwean Selfefficae 
Mean Score (SD) Mean Score (SD) Mean Score (SD) y 
and Expectancy-value 
Total 3.61(.81) 3.64(.88) 3.54(.95) 
Mathematics 3.61(.81) 3.66(.92) 3.45(1.02) 374" 
Science 181 3.52(.81) 3.55(.88) 3.50(.91) 628"* 
eel 3.69(.81) 3.70(.85) 3,68(.90) 742" 
F 2.14 1.37 2.84 
p-value 118 255 059 
n° 008 005 010 
“p<.01 


Table 3 illustrates the descriptive statistics of students’ performances on the sub-dimensions of students’ 


STEM attitudes. The mean (M) and standard deviation (SD) of the scores are presented, along with the results 
of analysis of variance (ANOVA) statistic. For the overall STEM attitudes, students achieve a little higher mean 
score in engineering/technology domain than those in the science and mathematics areas. As shown in table 3, 
students’ performance in engineering/technology domain has the highest construct average (M=3.69, SD=.81), 
followed by the mathematics domain (M=3.61, SD=.81), while student performance in science domain has the 
lowest average (M=3.52, SD=.81). However, no significant effect was detected on STEM domains for overall STEM 
attitudes (F(2,540)=2.14, p=.118, n° =.008). Although students performed alittle better in engineering/technol- 
ogy domain than in both mathematics and science domains, there were no detected significant effects for the 
self-efficacy category (F(2,540) =1.37, p=.255, n° =.005) and for the expectancy-value category (F(2,540)=2.84, 
p=.059, n° =.010) among three different sub-dimensions of STEM areas. 

Another aim of this research was to examine the correlation between self-efficacy belief and expectancy- 
value belief in STEM academic domains. In order to explore whether there was a correlation between primary 
students’ STEM self-efficacy and expectancy-value beliefs, a liner correlation analysis was applied. From the results 
of the correlation test in Table 3, for the science attitudes, there was a moderate correlation between self-efficacy 
and expectancy-value beliefs (r=.628”). Also, a significant positive correlation existed between self-efficacy 
and expectancy-value beliefs for attitudes towards engineering/technology domain (r=.742”). In addition, the 
correlation between mathematics self-efficacy and expectancy-value beliefs was much more weaker (r=.374"). 


Differences of Students’ STEM Domain-specific Attitudes in terms of Genders or Grade levels 


The second goal of the research was to explore whether primary students exhibited different attitudes 
towards various STEM domains among groups in terms of genders or grade levels. Thus, students’ performances 
were compared not only on the overall STEM attitude, but also on both the self-efficacy category and the 
expectancy-value category according to gender and grade level. In terms of grade levels, as introduced in the 
previous research (Zhou et al., 2019), students in the primary level were categorized as lower primary group with 
students from grade one to grade three and upper primary group with students from grade four to grade six. 
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Table 4 
The Mean Score for Each Group of Students and the T-test Statistic Results 


N STEM attitudes Self-efficacy Expectancy-value 
Mean Score (SD) Mean Score (SD) Mean Score (SD) 
Lower Lower Lower Upper 
Girls primary Girls primary Girls primary primary 
group group group 
Overall 3.52(.53) — 3.71(.59) 3.53 (85) 3.74 (.87) 3.53.91) 3.62(.96) 3.48.94) 
t(p) 1.25(.211) 2.07(.040) 1,86(.064) 2.26(.024) 141(.888) 1.75(.080) 
Mathematics 3.69(.83)  3.43(.73) —3.76(.73) 3.49/85) 3.78.94) 3.40(.84) 3.85/83) —3.52(.97) — 3.43(1.04) 3.51.99) 3.51(1.06) — 3.41(1.00) 
t(p) 2.02(.045) 2.32(.022) 2.55(.012) 2.42(.016) -480(.631) 627(.531) 
181 
Science 3.52(.84)  3.49(.76) —3.57(.84) 3.47/79) -3.54(.87) 3.58/90) 3.54.95) 3.57/81) 3.51/95) 3.45.82) 3.58/95) 3.43.88) 
t(p) 227(.821) .769(.443) ~.259(.796) -.267(.790) A57(.649) 1.15(.250) 
Engineering/ 
Technology 3.72(:82)  3.63(.80) —3.80(.74) 3.60/86) 3.74(.87) 3.62(.80) 3.82(.79) — 3.60(.89) — 3.69(.90) 3.64.91) 3.77/85) — 3.60/.94) 
t(p) 688(.492) 1.66(.099) 871(.385) 1,75(.082) .368(.714) 1,32(.190) 


Table 4 presents the mean score for each group of students and the results of t-test statistic. In terms of 
genders, there was no significant difference between boy students and girl students in the overall STEM at- 
titudes, the overall self-efficacy beliefs, and the overall expectancy-value beliefs, regardless of different STEM 
domains. While in terms of grade levels, lower primary students performed significantly better than upper 
primary students not only in the overall self-efficacy category (t=2.26, df=179, p=.024), but also in the overall 
STEM attitude (t=2.07, df=179, p=.040), with the later result clearly identified in the previous result (Zhou et al., 
2019). However, in the overall expectancy-value category, although lower primary students (M=3.62, SD=.96) 
achieved a little higher mean score than upper primary students (M=3.48, SD=.94), no statistically significant 
difference was consistently detected between the two groups (t=1.75, df=179, p=.080). 

In regard to student attitudes towards various STEM domains, the statistical data in science and engineer- 
ing/technology domains indicated that there was no significant difference either between boy students and 
girl students or between lower primary students and upper primary students in the overall STEM attitude, the 
overall self-efficacy category, and the overall expectancy-value category. 

However, considering the mathematics attitude, the statistical value indicated that there was a significant 
difference (t=2.02, df=179, p=.045) between boy students and girl students, with better performance of boy 
students (M=3.69, SD=.83) than that of girl students (V=3.43, SD=.73). Meanwhile, a significant effect was 
detected in terms of grade levels of mathematics attitude (t=2.32, df=179, p=.022), indicating that students in 
lower primary group (M=3.76, SD=.73) performed much better than students in upper primary group (M=3.49, 
SD=.85). Obviously, the differences were mainly reflected in the mathematics self-efficacy beliefs among groups 
in terms of genders (t=2.55, df=179, p=.012; boys: M=3.78, SD=.94; girls: M@=3.40, SD=.84) and grade levels (t=2.42, 
df=179, p=.016; lower: M=3.85, SD=.83; upper: M=3.52, SD=.97), with higher scores of boy students and students 
in the lower primary group. According to the analysis, boy students were more confident in mathematics than 
girl students, as well as students in lower primary group showed more confidence in mathematics than those in 
upper primary group. Besides, no statistical significance was detected in mathematics expectancy-value beliefs, 
regardless of gender or grade level. 
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Discussion on Students’ STEM Domain-specific Self-efficacy and Expectancy-value Beliefs 


Given that most studies have concentrated in assessing students’ overall STEM attitudes or the attitudes to- 
wards individual STEM discipline, the present research aimed to address some of the gap in the literature review by 
surveying primary school students’ attitudes towards various STEM domains including science, mathematics, and 
engineering/technology. The results suggest that there is no significant effect not only on overall STEM attitudes 
but also on both the overall self-efficacy beliefs and overall expectancy-value beliefs among these three domains for 
primary school students. It has been mentioned above that little research has focused on STEM attitudes involving 
primary students, especially the lower graders (Zhou et al., 2019). In the literature database, there is a small part of 
studies focusing on middle school students’ STEM attitudes (Wiebe et al., 2018; Unfried et al., 2014), while a large 
part of studies concentrate on measuring STEM attitudes of high school students and college students (Fraser, 1978; 
Simpkins et al., 2006; Speering & Rennie, 1996; Wang, 2012; 2013), possibly because the latter population is directly 
related to career choice after graduation. As revealed in previous studies, middle-school students considered sci- 
ence more difficult and unpleasant than other STEM domains (Mooney & Laubach, 2002). As well, another research 
found that middle school students had less career interest in engineering than mathematics and science (Lederman 
& Lederman, 2013).When comparing the present results of STEM domain-specific attitudes among primary school 
students with the related results in previous studies involved other graders, inconsistencies are found. It is impor- 
tant to note in the present research that students in primary school showed no difference on both the self-efficacy 
beliefs and expectancy-value beliefs. After further analyzing the career interest within the expectancy-value beliefs 
of mathematics (item 2:“When | am older, | might choose a job that uses math”), science (item 10:“I might choose a 
career in science”), and engineering/technology (item 26; “I believe | can be successful in engineering’) , it is found 
that there is a significant effect (F(2,540)=4.86, p=.008, 7-01 8) among different STEM subjects in terms of career 
interest, with lower mean scores in math domain (M=3.24, SD=1.24) and science domain (M=3.39, SD=1.36), and 
a higher mean score in engineering/technology domain (M=3.66, SD=1.23). Post-hoc comparisons identified that 
student's career interest in engineering/technology domain is significantly higher than math domain and science 
domain. In other words, the engineering/technology area is the most popular among primary school students. The 
possible reason is that mathematics and science have long been regarded as the most fundamental disciplines, 
belonging to the accumulation and reserve of basic knowledge of other disciplines; engineering and technology, 
on the other hand, are highly technical fields, which are popular career choices in China. 

From the perspective of the relationship between primary students’ STEM self-efficacy and expectancy-value 
beliefs, there are different correlations between these two categories of STEM attitudes regarding to different STEM 
domains. Most encouraging, self-efficacy and expectancy-value beliefs are strongly correlated for engineering/ 
technology domain. Moreover, a borderline strong correlation is shown between self-efficacy and expectancy-value 
beliefs for science domain. Note that a weaker correlation for mathematics suggests that students’ mathematics 
expectancy-value is not strongly associated with their self-efficacy beliefs comparing to science and engineering/ 
technology. In comparison, having been reported in the previous studies, there were moderate or even negative 
correlations among various variables of STEM attitudes (Johnpaul et al., 2018). Also, little relation has been found 
between self-efficacy and outcome expectancies in STEM-related domains (Maddux et al., 1986). However, based 
on the evidence of the positive correlations in the present research, it appears to have strong association between 
self-efficacy and expectancy-value in the science domain and engineering/technology domain in the primary 
population. It suggests that primary students are more likely to show expectancy-value in STEM fields if they have 
success in science or high self-efficacy in engineering/technology. These results provide evidence to support that 
STEM self-efficacy could be considered as a predictor of expectancy-value (Fouad & Smith, 1996) and it is in line 
with the viewpoint that having high expectancy-value beliefs is associated with the student’s STEM academic suc- 
cess (Scott & Mallinckrodt, 2005), mainly in science and engineering/technology domains. 


Discussion on the Differences of Students’ STEM Domain-specific Attitudes in Terms of Genders or Grade Levels 
As demonstrated in the previous research, there was no gender difference in students’ overall STEM attitudes, 


but lower primary students exhibited better STEM attitudes than upper primary students (Zhou et al., 2019). 
The current research is a more progressive analysis of student performances on STEM domain-specific attitudes. 
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As evident from the data, the grade difference in overall STEM attitude was contributed by grade difference in 
mathematics attitude, as primary students did not perform significant difference in both science attitude and 
engineering/technology attitude in terms of grade level. Consistently, the grade difference in overall self-efficacy 
beliefs was only caused by grade difference in mathematics self-efficacy beliefs. As well, although no significant 
difference was found between boy students and girl students in the overall STEM attitudes and the self-efficacy 
beliefs, gender difference still remained significant in mathematics overall attitudes and self-efficacy beliefs. This 
finding has been confirmed in the previous literature reviews (Eddy & Brownell, 2016; Cheryan et al., 2017). 

However, as detailed earlier in the review of grade difference of various STEM disciplines attitudes, there was 
a general decline in science and engineering/technology attitudes as school progress for elder students (Ali et 
al., 2013; Potvin & Hasni, 2014a; Said et al., 2016), but mathematical attitude was relatively stable over time from 
upper primary school through higher grades (Unfried et al., 2014). It is worth noting that the present result in 
primary school population is inconsistent with, or even contrary to, the previous findings in the elder group. The 
underlying reason for the differences in STEM domain-specific attitudes may be the different teaching standards 
and assessment standards set by the Ministry of Education (MoE) for various STEM disciplines in primary school in 
China. Mathematics, along with Chinese and English, is positioned as the key discipline for teaching assessment 
at each school year and selection of outstanding students at the end of primary school. With the growth of grade, 
the increasing learning difficulty of mathematics might be directly associated to the decline of students’ attitude 
of mathematics. Whereas, science and engineering/technology are relatively inferior disciplines, which are not 
included in the assessment and selection requirements from grade 1 through grade 6 in primary school. Therefore, 
students’ attitudes towards science and engineering/technology are comparatively stable throughout the primary 
school years, as most students consider these disciplines to be less formal. In addition, gender had an impact on 
primary students’ mathematics attitude but not their science and engineering/technology attitudes. The analysis of 
gender and grade differences in STEM domain-specific attitudes among primary students, which was not covered 
in the previous literature (Zhou et al., 2019; Unfried et al., 2014), may serve as an extension of the related research. 

In the aspect of expectancy-value beliefs of STEM attitudes, it is found that primary students showed no differ- 
ent value in their expectancy beliefs across genders and grade levels in various STEM domains. Although lower and 
upper primary students showed different levels of self-efficacy beliefs, they did not differ in how much they value 
STEM domains. For genders, the present findings support the view of no gender differences in the expectancy- 
value levels of STEM (Eccles, 2009). Especially in mathematics, students’ different senses of self-efficacy in terms of 
genders or grade levels did not influence their mathematics expectancy-value beliefs. However, it has been pointed 
out that boy students achieved higher level STEM expectancy-value beliefs as school progress (Watt et al., 2012; 
Yee & Eccles, 1988), Thus, it still needs to advocate that both self-efficacy and expectancy-value should deserve 
enough attention. It has been identified in most studies that the majority of students who major in STEM fields 
were directly affected by their growing STEM attitudes in late childhood and early adolescence (Wang 2013). The 
formation and cultivation of students’ STEM self-efficacy and expectancy-value in the early school are associated 
to their later STEM degrees and STEM career aspirations. 


Conclusions 


The present analysis extends the previous research to gender and grade differences of students’ STEM 
domain-specific attitudes including self-efficacy beliefs and expectancy-value beliefs in primary school years, 
and their correlations. On the one hand, the results suggest there are no significant effects among these three 
STEM domains in the overall attitudes, the overall self-efficacy beliefs, and the overall expectancy-value beliefs 
for primary students. The correlations between self-efficacy and expectancy-value are much stronger for the sci- 
ence domain and engineering/technology domain than the mathematics domain. On the other hand, no gender 
difference of the self-efficacy beliefs was detected except in the mathematics domain, and the result that lower 
primary students performed significantly better than upper primary students in the self-efficacy was also mainly 
contributed by the grade difference in the mathematics domain. However, no different expectancy-value beliefs 
existed across genders and grade levels in various STEM domains. 

The present research reported some inconsistent results in the primary school year with those in the elder 
group. For example, it was addressed in the previous result that middle-school students considered science more 
difficult and unpleasant than other STEM domains, but no significant effect has been found among STEM domain- 
specific attitudes in primary school. For another example, it has been pointed out that boy students achieved higher 
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level STEM expectancy-value beliefs as school progress, whereas the present finding showed no gender differences 
in STEM expectancy-value levels across genders for primary students. In addition, as detailed earlier in the review, 
there was a stable mathematical attitude and a general decline in science and engineering/technology attitudes 
as grade grow, while the grade difference of STEM attitudes was only caused by grade difference in mathematics 
attitude in the present research. Except for the underlying reasons towards the STEM educational reality in China 
that has been addressed above, more attention should be paid to the primary school population, especially the 
lower graders, because the period of primary school plays an essential role in their future choices of STEM courses 
and their interests in STEM careers. 

Obviously, the present research also has its limitations. For example, the participants of this research were 
from a primary school in a province of China, and there is a lack of data collection in a wider scope to enhance 
the credibility of the research conclusions. In addition, the research did not track and record the performances of 
primary school students’ STEM domain-specific self-efficacy and expectancy-value beliefs with the development 
of grade in a long-term way, therefore it is impossible to know how the STEM domain-specific attitudes of the 
same participant groups change with the increase of age. These limitations are expected to be fully explored and 
improved in future research. 


Acknowledgements 


The research is supported in part by 2020 Guangzhou Philosophy and Social Science Planning Fund of PR. 
China under the Grant No. 2020GZQN20. Any opinions, findings, and conclusions or recommendations expressed 
in this paper are those of the authors and do not necessarily reflect the views of the funding agencies. 


Declaration of Interest 
Authors declare no competing interest. 


References 


Adams, W. K., Perkins, K. K., Podolefsky, N. S., Dubson, M., Finkelstein, N. D., & Wieman, C. E. (2006). New instrument for measuring 
student beliefs about physics and learning physics: The Colorado learning attitudes about science survey. Physical Review 
Special Topics - Physics Education Research, 2(1), Article 010101. https://doi.org/10.1103/PhysRevSTPER.2.010101 

Aiken, L. R. (1970). Attitudes toward mathematics. Review of Educational Research, 40(4), 551-596. 

Akgunduz, D. (2016). A research about the placement of the top thousand students in STEM fields in Turkey between 2000 and 2014. 
Eurasia Journal of Mathematics, Science & Technology Education, 12 (5), 1365-1377. https://doi.org/10.12973/eurasia.2016.1518a 

Akpinar, E., Yildiz, E., Tatar, N., & Ergin, O. (2009). Students’ attitudes toward science and technology: an investigation 
of gender, grade level, and academic achievement. Procedia - Social and Behavioral Sciences, 1(1), 2804-2808. 
https://doi.org/10.1016/j.sbspro.2009.01.498 

Ali, M. M., Yager, R. E., Hacieminoglu, E., & Caliskan, I. (2013). Changes in student attitudes regarding science when taught by 
teachers without experiences with a model professional development program. School Science and Mathematics, 113(3), 
109-119. https://doi.org/10.1111/ssm.12008 

Atkinson, R. (2013). A short and long-term solution to America’s STEM crisis. The Hill. http://thehill.com/blogs/congress-blog/ 
technology/287435-a-short-andlong-term-solution-to-americas-stem-crisis 

Bandura, A. (1997). Self-efficacy: the exercise of control. Freeman. https://doi.org/10.1891/0889-8391.13.2.158 

Boone, W. J. (1997). Science attitudes of selected middle school students in China: A preliminary investigation of. School Science 
& Mathematics, 97(2), 96. https://doi.org/10.1111/j.1949-8594.1997.tb17349.x 

Business-Higher Education Forum. (2010). Increasing the number of STEM graduates: Insights from the U.S. STEM education modeling 
project. Author. 

Carnevale, A. P., Smith, N., & Melton, M. (2011). STEM: Science, technology, engineering, mathematics. Georgetown University Center 
on Education and the Workforce. https://files.eric.ed.gov/fulltext/ED525307.pdf 

Chen, G., Gully, S. M., Whiteman, J.-A., & Kilcullen, R. N. (2000). Examination of relationships among trait-like individual 
differences, state-like individual differences, and learning performance. Journal of Applied Psychology, 85(6), 835-847. 
https://doi.org/10.1037/0021-9010.85.6.835 

Cheryan, S. S., Ziegler, A., & Montoya, A. K., & Jiang, L. (2017). Why are some stem fields more gender balanced than others? 
Psychol. Bull, 143, 1. https://doi.org/10.1037/bul0000052 

Christensen, R., Knezek, G., & Tyler-Wood, T. (2014). Student perceptions of science, technology, engineering and mathematics 
(stem) content and careers. Computers in Human Behavior, 34, 173-186. https://doi.org/10.1016/j.chb.2014.01.046 

Corlu, M. S., Capraro, R. M., & Capraro, M. M. (2014). Introducing STEM education: Implications for educating our teachers in the 
age of innovation. Education and Science, 39 (171), 74-85. https://core.ac.uk/download/pdf/52924035.pdf 


687 


https://doi.org/10.33225/jbse/21.20.677 LTT 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


PRIMARY STUDENTS’ PERFORMANGE OF STEM DOMAIN-SPEGIFIG SELF-EFFIGAGCY BELIEF ISSN 1648-3898 print 
AND EXPECTANGY-VALUE BELIEF 
(PP. 677-690) ISSN 2538-7138 /Online/ 


Drew, C. (2011).Why science majors change their minds (It’s Just So Darn Hard). The New York Times, November 4, (pp. 1-6). https:// 
www.researchgate.net/publication/266605573 Why Science Majors Change Their Minds It’s Just So Darn Hard 

Eccles, J. (2009). Who am | and what am | going to do with my life? Personal and collective identities as motivators of action. 
Educational Psychologist, 44, 78-89. https://doi.org/10.1080/00461520902832368 

Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109-132. 
https://doi.org/10.1146/annurev.psych.53.100901.135153 

Eddy, S. L. & Brownell, S. E. (2016). Beneath the numbers: A review of gender disparities in undergraduate education across 
science, technology, engineering, and math disciplines. Physical Review Physics Education Research, 12, Article 020106. 
https://doi.org/10.1103/PhysRevPhysEducRes.12.020106 

Ergun, A. (2019). Identification of the interest of Turkish middle school students in stem careers: Gender and grade level differences. 
Journal of Baltic Science Education, 18(1), 90-104. https://doi.org/10.33225/jbse/19.18.90 

Erkut, S., & Marx, F. (2005). 4 schools for WIE (Evaluation report). Wellesley College, Center for Research on Women. http://www. 
coe.neu.edu/Groups/stemteams/evaluation.pdf 

Fan, W. (2011). Social influences, school motivation and gender differences: An application of the expectancy-value theory. 
Educational Psychology, 31, 157-175. https://doi.org/10.1080/01443410.2010.536525 

Fouad, N. A., & Smith, P. L. (1996). A test of a social cognitive model for middle school students: Math and science. Journal of 
Counseling Psychology, 43(3), 338-346. https://doi.org/10.1037/0022-0167.43.3.338 

Fraser, B. J. (1978). Development of a test of science-related attitudes. Science Education, 62, 509-515. 
https://doi.org/10.1002/sce.3730620411 

Gardner, P.L. (1975). Attitudes to science: a review. Studies in Science Education, 2, 1-41.https://doi.org/10.1080/03057267508559818 

Holdren, J. P., & Lander, E. S. (2012). Engage to excel: Producing one million additional college graduates with degrees in science, 
technology, engineering, and mathematics (Executive Report). President’s Council of Advisors on Science and Technology. 
https://doi.org/10.1001/jama.2014.8906 

Jarvis, T., & Pell, A. (2005). Factors influencing primary school children’s attitudes toward science before, during, and after a visit 
to the UK National Space Centre. Journal of Research in Science Teaching, 42(1), 53-83. https://doi.org/10.1002/tea.20045 

Johnpaul, K., Frances, Q., & Terry, L. (2018). The keys to STEM: Australian year 7 students’ attitudes and intentions towards science, 
mathematics and technology courses. Research in Science Education, 1-28. https://doi.org/10.1007/s11165-018-9754-3 

Jones, G. M., Howe, A., & Rua, M. J. (2000). Gender differences in students’ experiences, interests and attitude toward science and 
scientists. Science Education, 84, 180-192. https://doi.org/10.1002/(SICI) 1098-237 X(200003)84:2<180::AID-SCE3>3.0.CO;2-X 

Kennedy, J., Quinn, F., & Lyons, T. (2018). Australian enrolment trends in technology and engineering: putting the 
T and E back into school STEM. International Journal of Technology and Design Education, 28(2), 553-571. 
https://doi.org/10.1007/s10798-016-9394-8 

Kizilay, E. (2018). Career and employment in STEM fields in Turkey. Journal of International Social Research, 11 (56), 570-574. 

Kristen, E., Jennifer, C., & Holly, Y. A. (2012). Designing evaluations for k-12 robotics education programs. In Robots in k-12 education: 
anew technology for learning (pp. 31-53). Information Science Reference. https://doi.org/10.4018/978-1-4666-0182-6.chO002 

Lederman, L. M. (2008). Scientists and 21st century science education. Technology in Society, 30(3), 397-400. 
https://doi.org/10.1016/j.techsoc.2008.04.014 

Lederman, N. G., & Lederman, J. S. (2013). Is it STEM or “S & M” that we truly love? Journal of Science Teacher Education, 24(8), 
1237-1240. https://doi.org/10.1007/s10972-013-9370-z 

Lent, R. W., Sheu, H. B., Singley, D., Schmidt, J. A., Schmidt, L. C., & Gloster, C. S. (2008). Longitudinal relations of self-efficacy to 
outcome expectations, interests, and major choice goals in engineering students. Journal of Vocational Behavior, 73(2), 
328-335. https://doi.org/10.1016/j.jvb.2008.07.005 

Luo, W., Wei, H. R., Ritzhaupt, A. D., Huggins-Manley, A. C., & Gardner-Mccune, C. (2019). Using the S-STEM survey to evaluate 
a middle school robotics learning environment: Validity evidence in a different context. Journal of Science Education and 
Technology, 28(4), 429-443. https://doi.org/10.1007/s10956-019-09773-z 

Maddux, J. E., Norton, L. W., & Stoltenberg, C. D. (1986). Self-efficacy expectancy, outcome expectancy, and outcome 
value: Relative effects on behavioral intentions. Journal of Personality and Social Psychology, 51, 783-789. 
https://doi.org/10.1037/0022-3514.51.4.783 

Manning, M. M., & Wright, T. L. (1983). Self-efficacy expectancies, outcome expectancies, and the persistence of pain control in 
childbirth. Journal of Personality & Social Psychology, 45(2), 421-31. http://doi.org/10.1037/0022-3514.45.2.421 

McMahon, M. M., & Showers, T. T. (2012). Partnering for Success in the 21st Century. In Advancing the STEM Agenda: 
Quality Improvement Supports STEM: Selected Papers from the 2011 Advancing the STEM Agenda in Education, the 
Workplace and Society Conference at the University of Wisconsin-Stout, July, 2011 (p. 29). ASQ Quality Press. 
http://asq.org/edu/201 1/06/innovation/partnering-for-success-in-the-21st-century.pdf 

Miller, R. B., Greene, B. A., Montalvo, G. P., Ravindran, B., & Nichols, J. D. (1996). Engagement in academic work: The role of learning 
goals, future consequences, pleasing others, and perceived ability. Contemporary Educational Psychology, 21(4), 388-422. 
https://doi.org/10.1006/ceps.1996.0028 

Minner, D., Ericson, E., Wu, S., & Martinez, A. (2012). Compendium of STEM Student Instruments PART Il: Measuring Students’ Content 
Knowledge, Reasoning Skills, and Psychological Attributes. Abt Associates. 

Moakler, M. & Kim, M. M. (2014). College major choice in STEM: Revisiting confidence and demographic factors. Career Development 
Quarterly, 62, 128-143. https://doi.org/10.1002/j.2161-0045.2014.00075.x 

Mooney, M.A.,& Laubach, T.A. (2002). Adventure engineering: A design centered, inquiry-based approach to middle grade science and 
mathematics education. Journal of Engineering Education, 91(3), 309-318. https://doi.org/10.1002/j.2168-9830.2002.tb00708.x 


688 


— Wess https://doi.org/10.33225/jbse/21.20.677 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


= . PRIMARY STUDENTS’ PERFORMANGE OF STEM DOMAIN-SPEGIFIC SELF-EFFICACY BELIEF 
ISSN 1648 3898 fant AND EXPECTANCY-VALUE BELIEF 
ISSN 2538-71 38 /Online/ (PP. 6'77-690) 


Multon, K. D., Brown, S. D., & Lent, R. W. (1991). Relation of self-efficacy beliefs to academic outcomes: A meta-analytic investigation. 
Journal of Counseling Psychology, 38(1), 30-38. https://doi.org/10.1037/0022-0167.38.1.30 

National Academy of Engineering. (2008). Grand challenges for engineering. National Academies Press. 
https://doi.org/10.1109/EMR.2009.4804341 

National Research Council. (2010). Exploring the intersection of science education and 21st century skills: A workshop summary. 
National Academies Press. https://doi.org/10.17226/12771 

National Science Board., (2012). Science and engineering indicators 2012. National Science Board. 

OECD. (2015). How is the global talent pool changing (2013, 2030)? Education Indicators in Focus, No. 31. Author. 
https://files.eric.ed.gov/fulltext/ED532709.pdf 

OECD, Organization for Economic Co-operation and Development. (2017). Turkey-country notes-training in 2017 at aglance: OECD 
indicators. http://www.oecd.org/education/skills-beyond-school/EAG2017CN-Turkey-Turkish.pdf 

Osborne J., & Dillon J., (2008), Science education in Europe: Critical reflections (A Report to the Nuffield Foundation). King’s College. 
https://www.researchgate.net/profile/Jonathan_Osborne2/publication/252404504_Science_Education_in_Europe_ 
Critical_Reflections/links/543 178e40cf277d58e982a32/Science-Education-in-Europe-Critical-Reflections.pdf 

Osborne, J., Simon, S., & Collins, S. (2003). Attitudes towards science: a review of the literature and its implications. /nternational 
Journal of Science Education, 25(9), 1049-1079. https://doi.org/10.1080/0950069032000032199 

PCAST, President’s Committee of Advisors on Science and Technology. (2010). Prepare and Inspire: K-12 Education in Science, 
Technology, Engineering, and Math (STEM) for America’s Future. Executive Office of the President. 

Peterman, K., Kermish-Allen, R., Knezek, G., Christensen, R., & Tyler-Wood, T. (2016). Measuring student career interest within 
the context of Technology-Enhanced STEM Projects: A cross-project comparison study based on the Career Interest 
Questionnaire. Journal of Science Education and Technology, 1-13. https://doi.org/10.1007/s10956-016-9617-5 

Potvin, P., & Hasni, A. (2014a). Analysis of the decline in interest towards school science and technology from grades 5 through 
11. Journal of Science Education and Technology, 23(6), 784-802. https://doi.org/10.1007/s10956-014-9512-x 

Potvin, P., & Hasni, A. (2014b). Interest, motivation and attitude towards science and technology at K-12 levels: A systematic review 
of 12 years of educational research. Studies in Science Education, 50(1), 85-129. https://doi.org/10.1080/03057267.2014.881626 

Reinhold, S., Holzberger, D., & Seidel, T. (2018). Encouraging a career in science: a research review of secondary schools’ effects 
on students’ STEM orientation. Studies in Science Education, 54(1), 1-35. https://doi.org/10.1080/03057267.2018.1442900 

Rothwell, J. (2013). The hidden STEM economy: The metropolitan policy program at Brookings. Brookings Institution. 
https://doi.org/10.1080/03057267.2018.1442900 

Said, Z., Summers, R., Abd-El-Khalick, F., & Wang, S. (2016). Attitudes toward science among grades 3 through 12 Arab students 
in Qatar: Findings from a cross-sectional national study. /nternational Journal of Science Education, 38(4), 621-643. 
https://doi.org/10.1080/09500693.2016.1156184 

Schunk, D. H. (1991). Self-efficacy and academic motivation. Educational Psychologist, 26(3/4), 207. 
https://doi.org/10.1080/00461520.1991.9653133 

Scott, A., & Mallinckrodt, B. (2005). Parental emotional support, science self-efficacy, and choice of science major in undergraduate 
women. Career Development Quarterly, 53, 263-273. https://doi.org/10.1002/j.2161-0045.2005.tb00995.x 

Shapiro, C. A., & Sax, L. J. (2011). Major selection and persistence for women in STEM. New Directions for Institutional Research, 
(152), 5-18. https://doi.org/10.1002/ir.404 

Simpkins, S. D., Davis-Kean, P. E., & Eccles, J. S. (2006). Math and science motivation: A longitudinal examination of the links 
between choices and beliefs. Developmental Psychology, 42, 70-83. https://doi.org/10.1037/0012-1649.42.1.70 

Simpson, R. D., & Oliver, J. S. (1990). A summary of major influences on attitude toward and achievement in science among 
adolescent students. Science Education, 74(1), 1-18. https://doi.org/10.1002/sce.3730740102 

Speering, W., & Rennie, L. (1996). Students’ perceptions about science: the impact of transition from primary to secondary school. 
Research in Science Education, 26(3), 283-298. https://doi.org/10.1007/BF02356940 

Toma, R.B., & Greca, I. M. (2018). The effect of integrative STEM instruction on primary students’ attitudes toward science. FURASIA 
Journal of Mathematics, Science and Technology Education, 14(4), 1383-1395. https://doi.org/10.29333/ejmste/83676 

Unfried, A., Faber, M., Stanhope, D. S., & Wiebe, E. (2015). The development and validation of a measure of student attitudes 
toward science, technology, engineering, and math (s-stem). Journal of Psychoeducational Assessment, 33(7), 622-639. 
https://doi.org/10.1177/0734282915571160 

Unfried, A., Faber, M., & Wiebe, E. (2014). Gender and student attitudes toward STEM. Presented at the AERA Annual Meeting, 
Philadelphia, PA. Retrieved from https://eval.fi.ncsu.edu/wp-content/uploads/2016/03/AERA-2014-Unfried-Faber-Wiebe.pdf. 

Wang, M.-T. (2012). Educational and career interests in math: A longitudinal examination of the links between classroom environment, 
motivational beliefs, and interests. Developmental Psychology, 48(6), 1643-1657. https://doi.org/10.1037/a0027247 

Wang, X. (2013). Why students choose STEM majors: Motivation, high school learning, and postsecondary context of support. 
American Educational Research Journal, 50(5), 1081-1121. https://doi.org/10.3102/000283 1213488622 

Greenfield, T. A. (1996). Gender, ethnicity, science achievement and attitudes. Journal of Research in Science Teaching, 33, 901-933. 

Watt, H. M.G., Shapka, J. D., Morris, Z. A., Durik, A. M., Keating, D.P., & Eccles, J. S. (2012). Gendered motivational processes affecting 
high school mathematics participation, educational aspirations, and career plans: A comparison of samples from Australia, 
Canada, and the United States. Developmental Psychology, 48, 1594 -1611. https://doi.org/10.1037/a0027838 

Wiebe, E., Unfried, A., & Faber, M. (2018). The Relationship of STEM Attitudes and Career Interest. Eurasia Journal of Mathematics, 
Science and Technology Education, 14(10), em1580. 

Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 
25(1), 68-68. https://doi.org/10.1007/BF02209024 


689 


https://doi.org/10.33225/jbse/21.20.677 LTT 


690 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


PRIMARY STUDENTS’ PERFORMANGE OF STEM DOMAIN-SPEGIFIG SELF-EFFIGAGCY BELIEF ISSN 1648-3898 print 
AND EXPECTANGY-VALUE BELIEF 
(PP. 677-690) ISSN 2538-7138 /Online/ 


Williams, K., Kurtek, K., & Sampson, V. (2011). The affective elements of science learning: A questionnaire to assess-and improve- 
student attitudes toward science. Science Teacher, 78(1), 40-45. https://www.mendeley.com/catalogue/e9973010-588b- 
3237-8de9-61a62307fa1f/ 

Xie, Y., & Reider, D. (2014). Integration of innovative technologies for enhancing students’ motivation for science learning and 
career. Journal of Science Education and Technology, 23(3), 370-380. https://doi.org/10.1007/s10956-013-9469-1 

Yee, D. K., & Eccles, J. S. (1988). Parent perceptions and attributions for children’s math achievement. Sex Roles, 19, 317-333. 
https://doi.org/10.1007/BF00289840 

Yerdelen, S., Kahraman, N., & Tas, Y. (2016). Low socioeconomic status students’ stem career interest in relation to gender, grade 
level, and STEM attitude. Journal of Turkish Science Education, 13(Special Issue). 

Zhou, S.N., Zeng, H., Xu, S. R., Chen, L. C., & Xiao, H. (2019). Exploring changes in primary students’ attitudes towards science, 
technology, engineering and mathematics (stem) across genders and grade levels. Journal of Baltic Science Education, 18(3), 
466-480. https://doi.org/10.33225/jbse/19.18.466 

Zimmerman, B. J., & Bandura, A. (1994). Impact of self-regulatory influences on writing course attainment. American Educational 
Research Journal, 31, 845-862. https://doi.org/10.3102/00028312031004845_ 


Received: January 20, 2021 Accepted: July 16, 2021 


Cite as: Zhou, S. N., Chen, L.-C., Xu, S.-R., Lu, C.-T., Li, Q.-Y., & Li, D.-A. (2021). Primary students’ performance of STEM 
domain-specific self-efficacy belief and expectancy-value belief. Journal of Baltic Science Education, 20(4), 677-690. 
https://doi.org/10.33225/jbse/21.20.677 


Shao-Na Zhou PhD, Associate Professor, School of Physics and Telecommunication 
Engineering, South China Normal University, Guangzhou 510006, 
China. 


E-mail: zhou.shaona@m.scnu.edu.cn 
ORCID: https://orcid.org/0000-0003-1455-5122 


Lu-Chang Chen MSc., Teacher, Pazhou Experimental School of Guangzhou Zhixin 
Middle School, Guangzhou 510000, China. 
E-mail: 244557813@qq.com 


Shao-Rui Xu MSc., Teacher, School of Electronics and Communication, 
Guangdong Mechanical & Electrical Polytechnic, Guangzhou 
510550, China. 
E-mail: jayxee@hotmail.co.uk 


Chu-Ting Lu MSc., School of Physics and Telecommunication Engineering, South 
China Normal University, Guangzhou 510006, China. 
E-mail: chutingjnu@163.com 


Qiu-ye Li MS., School of Physics and Telecommunication Engineering, South 
China Normal University, Guangzhou 510006, China. 
E-mail: 2019021855@m.scnu.edu.cn 


De-An Li MSc., Associate Professor, School of Physics and Telecommunication 
(Corresponding author) Engineering, South China Normal University, Guangzhou 510006, 
China. 


E-mail: lidean@scnu.edu.cn 


— Wess" https://doi.org/10.33225/jbse/21.20.677 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


ISSN 1648-3898 | print 
ISSN 2538-7138 sontine/ 


INFORMATION 
FOR CONTRIBUTORS 


EDITORIAL AND REVIEW PROCEDURES 


Journal of Baltic Science Education (JBSE) publishes original scientific research articles in the field of Natural Science Education and 
related areas for all educational levels in the Baltic and other countries. It is possible to publish special (thematic) issues of JBSE. The 
papers should be submitted and will be published in English. JBSE will promote to establish contacts between researchers and practical 
educators both in the Baltic countries and countries around. 

The authors of the manuscripts are responsible for the scientific content and novelty of the research materials. Articles, published 
before in other international journals or papers’ collections will not be accepted for publication in JBSE. 

As a publication that represents a variety of cross-disciplinary interests, both theoretical and practical, the JBSE invites manuscripts on 
a wide range of topics, especially in the following areas: 


+ Didactics of natural sciences. + Philosophical, political, economical and social aspects 
+ Theory and practice in natural science teacher of natural science education. 
education. + The supplementary natural science education. 
+ Integrated natural science education. + ICT in natural science education. 
+ Natural science and technological literacy. + The standardisation of natural science education etc. 


+ General and professional natural science education. 


MANUSCRIPTS GUIDELINES 


The structure of the research paper presented to the Journal of Baltic Science Education should be as follows: abstract - short report of the 
investigation; introduction inc. aim and subject of the research; research methodologies and methods; results of the research incl. discussion; 
conclusions; list of references in APA style. 

The papers should be submitted in English. If English is a second language for the author, please consider having the manuscript 
proof read and edited before submitting. The preliminary text of the article can be sent as a.doc file in the attachment by e-mail: mail. 
jbse@gmail.com 


The text must be elaborated in Word for Windows, using 12 point Times New Roman letters. An article should not exceed 7-10 A4 pages, 
included figures, tables and bibliography. Publishing of longer articles should be negotiated separately. Texts margins: top and bottom 20mm, 
left - 25mm, right - 20mm. The title: capital letters, 14pt, bold; space between the title and the author's name is one line interval. Author's name 
and surname: small letters, 12pt, bold. Under the name, institution: 11 pt, italics; space between the title and the text: 1 line interval. Abstract 
- about 100-150 words - precedes the text. The text: 12pt Single or Auto spacing, in one column. Key words: no more than five words. The 
language must be clear and accurate. The authors have to present the results, propositions and conclusions in a form that can suit scientists 
from different countries. 


Titles of the tables and figures: 11 pt, small letters. Space between figures or tables and the text: 1 line interval. Introduction, titles of 
chapters and subchapters: 12pt, bold, small letters. Numbers: Arabic, subchapters numbered by two figures (1.1, 1.2, etc.). Figures, tables and 
captions should be inserted within the manuscript at their appropriate locations. Diagrams and graphs should be provided as finished black 
and white line artwork or electronic images. When there are a number of illustrations, the author should endeavour to reduce the amount of 
text to accommodate the illustrations in the limited space available for any article. 


References in the text should be presented in brackets (Knox, 1988; Martin, 1995). If necessary, the page can be indicated: (Martin, 
1995, p.48). The list of references should be presented after the text. The Words List of References: 11pt, bold, small letters. The references 
should be listed in full at the end of the paper in the following standard form: 

For books: Saxe, G.B. (1991). Cultural and Cognitive Development: Studies in Mathematical Understanding. Hillsdale, NJ: Erloaum. 

For articles: Bekerian, D.A. (1993). In Search of the Typical Eyewitness. American Psychologist, 48, 574-576. 

For chapters within books; Bjork, R.A. (1989). Retrieval Inhibition as an Adaptive Mechanism in Human Memory. In: H.L. Roediger Ill & F.1.M. 

Craik (Eds.), Varieties of Memory & Consciousness (pp. 309-330). Hillsdale, NJ: Erlbaum. 


On a separate page, author - related data should be presented in English: name, surname, degree and academic title, institution, full 
correspondence address in the clearest and most complete form /ordinary post and e-mail addresses /, position (to ensure anonymity in the 
review process). The author (authors) should confirm in writing, that the manuscript has not been published in other journal or 
handed over (transferred) to other journal for publication. 


EDITORIAL AND REVIEW PROCEDURES 


Manuscripts will be sent anonymously to reviewers with expertise in the appropriate area. All manuscripts will be rewieved by two 
experts before JBSE’s accept them for publication. This process usually takes about two months. The journal co-editors will make minor 
editorial changes; major changes will be made by the author(s) prior to publication if necessary. JBSE’s redaction will sent to author(s) 
only one correcture which must be sent back within 2 weeks. JBSE will not review submissions previously published elsewhere through 
print or electronic medium. 

Manuscripts submitted to the JBSE cannot be returned to authors. Authors should be sure to keep a copy for themselves. Authors’ 
signatures should be at the end of the paper and its second checked proofs. 

Manuscripts, editorial correspondence (and other correspondence for subscription and exchange), and any questions should be 
sent to editor-in-chief or to regional redactors. 


Journal's requirements for the authors are available online: http://www.scientiasocialis.|t/jbse/files/JBSE_requirements_2019.pdf 


691 


—<LL 


Journal of Baltic Science Education, Vol. 20, No. 4, 2021 


ISSN 1648-3898 /Print/, ISSN 2538-7138 /Online/ 


Compiled by: Vincentas Lamanauskas 
Linguistic Editor: Ilona Ratkeviciené 
Cover design by: Jurgina Jankauskiené 

Layout design by: Linas Janonis 


20 August 2021. Publishing in Quires 10,5. Edition 90 


Publisher Scientia Socialis Ltd., 
Donelaicio Street 29, LT-78115 Siauliai, Lithuania 
E-mail: scientia@scientiasocialis.It 
Phone: +370 687 95668 
http://www.scientiasocialis.lt 


Printing Siauliai printing house 
9A P. Luksio Street 
LT-76207 Siauliai, Lithuania 
Phone: +370 41 500 333. 
Fax: +370 41 500 336 
E-mail: info@dailu.lt 
https://siauliuspaustuve.|t/ 


